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Abstract 


During  1995-1998,  Xerox  Corporation’s  West  Coast  Production  Systems  Group  (PSG  West) 
worked  with  the  Software  Engineering  Institute  (SEI)  to  apply  the  prototype  Process  Change 
Model  (PCM)  to  aid  in  their  efforts  to  reach  Level  2  of  the  Software  Capability  Maturity 
Model®  (CMM) ,®  and  to  develop  the  generic  processes  required  for  Level  3.  The  Process 
Change  Model,  along  with  a  companion  guidebook,  was  designed  to  provide  the  basis  for  a 
systematic  approach  to  technology-specific  change  based  in  part  on  “whole  product”  princi¬ 
ples,  with  a  focus  on  one  key  process  area  (KPA)  at  a  time.  This  report  describes  a  collabora¬ 
tive  effort  to  develop  a  more  systematic  and  detailed  approach  to  software  process  improve¬ 
ment  (SPI)  through  use  and  evaluation  of  prototype  versions  of  the  PCM  and  guidebook.  In 
particular,  the  work  of  the  PSG  West  software  engineering  process  group  (SEPG)  to  apply  the 
PCM  and  guidebook  in  working  with  improvement  action  teams  focused  on  the  KPAs  of  the 
Software  CMM  is  described.  Lessons  learned  about  the  “live”  evaluation  and  maturation  of  a 
new  process  and  guidebook  such  as  this  are  presented.  These  lessons  should  be  of  interest  to 
those  engaged  in  work  on  technology  maturation  and  the  adoption  of  technological  or  process 
innovations  as  well  as  to  those  engaged  in  SPI  and  process  development. 


®Capability  Maturity  Model  and  CMM  are  registered  in  the  U.S.  Patent  Office. 
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1  Introduction  and  Background 


Xerox  Corporation’s  West  Coast  Production  Systems  Group  (PSG  West),  an  organization  that 
includes  about  450  software  developers,  initiated  a  Software  Capability  Maturity  Model 
(CMM)-based  software  process  improvement  effort  beginning  with  a  CMM-based  appraisal  for 
internal  process  improvement  (CBAIPI)  in  late  1994  [Paulk  1991]. 1  From  1995  to  mid-1998 
PSG  West  created  a  software  process  improvement  (SPI)  strategy  and  executed  related  tasks  for 
its  dozen  or  so  software  projects.  Four  improvement  action  teams  (IATs)  were  initiated  over  a 
three-month  period,  with  the  goal  of  achieving  Software  CMM  Maturity  Level  2  and  of  creat¬ 
ing  the  process  solutions  that  would  be  required  for  Level  3. 

PSG  West  established  a  collaboration  with  the  Software  Engineering  Institute  (SEI)  to  use  the 
SEI  prototype  Process  Change  Model  (PCM)  and  related  guidebook.2  The  goal  of  the  col¬ 
laboration  was  to  provide  structure  to  the  work  of  PSG  West  improvement  action  teams.  The 
SEI  would  receive  the  feedback  necessary  for  the  PCM’s  further  development.  Because  of 
Xerox’s  quality-based  culture  (divisions  of  Xerox  Corporation  won  the  Malcolm  Baldrige 
award  in  1989  and  again  in  1997)[Caudron  1991],  PSG  West  approached  the  SPI  effort  sys¬ 
tematically,  documenting  effort  expended,  minutes  of  meetings,  and  lessons  learned  in  a  dis¬ 
ciplined  manner,  and  making  available  a  good  record  of  the  collaboration. 

This  report  describes  the  process  and  results  of  the  collaboration  during  Xerox’s  1995-1998 
SPI  work  (their  work  continues).  The  strategy  inherent  in  the  PCM  reflected  an  approach  to 
software  process  improvement  that  is  common  in  the  software  community — that  is,  a  key 
process  area  (KPA)-based  approach.  The  emphasis  in  this  approach  is  on  the  development  of 
“generic”  process  descriptions,  by  IATs,  which  can  then  be  adapted  by  project  organizations.3 
This  report  describes  the  pros  and  cons  of  that  strategy  as  applied  at  PSG  West,  in  the  context 
of  a  detailed  look  at  the  collaboration. 


1  We  reference  the  original  technical  report  on  the  Software  CMM,  not  the  current  and  more  widely 
known  book  on  the  Software  CMM,  because  the  original  is  what  was  used  at  PSG  West.  The  current 
version  is  [Paulk  1995]. 

2  We  refer  to  both  the  model  and  guidebook  from  this  point  forward  as  the  “PCM”  or  the  “prototype 
PCM,”  as  was  the  case  in  PSG  West’s  usage  of  the  terms.  Throughout  this  report,  we  emphasize  that 
the  prototype  versions  of  the  PCM  itself  and  the  related  guidebook  were  used.  Since  the  work 
described  here  was  completed,  the  model  has  been  reengineered  both  for  SPI  use  [Kasunic  1998]  and 
for  application  to  more  general  technology  adoption.  The  latter  version  was  renamed  the  “technology 
transition  model,”  and  is  presented  in  the  SEI  course,  Introducing  New  Software  Technology.  In 
addition,  aspects  of  the  PCM  have  been  incorporated  into  the  IDEAL-Based  New  Technology  Rollout 
(INTRo)  process,  now  being  evaluated  [Levine  1999]. 

3  This  is  a  typical  approach  to  Software  CMM-based  software  process  improvement;  see  for  example, 
any  of  the  proceedings  of  the  Software  Engineering  Process  Group  conference,  1996-1999. 
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This  report  is  organized  as  follows:  In  the  remainder  of  this  chapter  we  describe  how  SPI  be¬ 
gan  at  PSG  West  and  give  background  on  the  SPI  infrastructure  and  the  decision  to  use  a 
model-based  approach  to  SPI.  Brief  descriptions  of  the  PCM  and  another  SEI  model  related 
to  SPI,  the  IDEALsm  model  [McFeeley  1996],  are  provided.  Chapters  Two  through  Nine  each 
describe  one  of  the  eight  major  activity  areas  from  the  PCM.  For  each  area,  we  describe  the 
purpose  of  the  activity  and  the  tasks  it  included,  and  the  highlights  of  what  PSG  West  actually 
did  (the  PCM  was  offered,  not  mandated,  for  use).  Then,  also  for  each  activity,  we  discuss 
issues  at  both  the  IAT  and  SEPG  level  and  lessons  learned.  In  our  discussion  of  Activity  8, 
Wrap  Up  (in  Chapter  9),  we  focus  on  overall  lessons  learned  in  the  use  of  the  PCM.  In  the 
final  chapter,  we  present  our  perception  of  the  value  of  the  collaboration.  The  experiences 
described  here  should  be  of  interest  to  those  engaged  in  work  on  technology  maturation  or 
the  adoption  of  technological  or  process  innovations,  as  well  as  to  those  engaged  in  SPI  and 
process  development. 

1.1  How  SPI  Began  at  PSG  West 

In  1986,  a  number  of  quality  improvement  teams  and  benchmarking  projects  began  to  docu¬ 
ment  the  limitations  of  existing  development  and  maintenance  practices  in  PSG  West  soft¬ 
ware  projects.  While  pockets  of  improvement  existed,  these  activities  did  not  spread  to  PSG 
West  as  a  whole.  PSG  West’s  approach  to  software  development  was  similar  to  other  typical 
organizations  assessed  at  Software  CMM  Level  1 .  Management  at  PSG  West  believed  that 
there  must  be  better  approaches  to  software  development,  despite  the  business  necessity  of 
giving  priority  to  schedule. 

Malcolm  Baldrige  Award  winners  are  required  to  “adopt”  a  university  and  teach  people  at 
that  university  quality  principles.  In  1993  Xerox  as  a  corporation  adopted  Carnegie  Mellon 
University  (CMU)  and  initiated  a  quality  education  effort.4  In  this  joint  program,  personnel 
from  the  Software  Engineering  Institute  at  CMU  and  Xerox  participated  in  technical  inter¬ 
change:  quality  principles  and  methods  were  provided  by  Xerox  to  CMU,  and  assistance  with 
Software  CMM-based  software  process  improvement  was  given  by  the  SEI  to  Xerox.  In 
1993,  Xerox  Corporation  embraced  the  Software  CMM  as  the  basis  for  diagnosis  and  im¬ 
provement  of  software  process  throughout  the  company,  and  began  a  working  relationship 
with  the  SEI,  which  provided  consultants  to  assist  in  a  number  of  SPI  efforts. 

1.2  Establishing  a  Permanent  SPI  Organizational 
Infrastructure 

In  SPI  orientation  activities  conducted  by  SEI  consultants,  management  at  PSG  West  agreed 
that  any  software  process  improvement  effort  must  be  continuous  and  long-term.  Thus  man¬ 
agement  at  PSG  West  determined  that  for  SPI  to  succeed,  PSG  West  needed  to  establish  a 
permanent  organizational  infrastructure  that  could  build  and  deploy  effective  process  im- 


SM  IDEAL  is  a  service  mark  of  Carnegie  Mellon  University. 
4  Paul  Allaire,  then  president  of  Xerox,  is  a  CMU  alumnus. 
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provements  through  direct  and  cohesive  participation  at  all  levels  of  management.  Based  on 
the  recommendations  of  the  SEI,  PSG  West  established  the  following  groups  to  be  directly 
responsible  for  software  process  improvement  at  PSG  West: 

•  Executive  council 

•  Management  steering  team  (MST) 

•  Software  engineering  process  group 

•  Improvement  action  teams 

In  making  these  suggestions,  the  SEI  consultants  followed  guidance  based  on  the  lessons 
learned  by  the  software  community  and  the  SEI;  these  lessons  were  later  described  in  IDEAL: 
A  User’s  Guide  for  Software  Process  Improvement  [McFeeley  1996].  These  early  SPI  efforts 
pre-dated  the  PCM  collaboration. 

1.3  How  PSG  West  Began  Using  PCM 

Details  of  the  PCM  were  taught  in  an  SEI  course.  Introducing  New  Software  Technology,  at¬ 
tended  by  the  PSG  West  SEPG.  At  that  point  in  initial  SPI  activities,  the  SEPG  was  investi¬ 
gating  approaches  to  IAT  work.  A  collaboration  with  the  SEI,  to  apply  the  PCM  in  the  work 
of  one  or  two  IATs  at  PSG  West,  was  subsequently  negotiated.  The  risk  of  using  the  new 
model  and  guidebook  was  to  be  mitigated  by  having  the  SEI  author  work  with  the  PSG  West 
SEPG  as  a  coach.5 

1.4  Using  Models  for  Guidance  in  SPI  at  PSG  West 

The  PSG  West  SEPG  chose  several  models  to  provide  a  basis  for  their  SPI  strategy  and  plans. 
SEPG  members  selected  the  Software  CMM  as  the  reference  model  for  improved  software 
engineering  practice.  PSG  West  used  the  SETs  IDEAL  model  because  it  provided  the  basis 
for  an  overall  strategy  and  its  guidebook  [McFeeley  1996]  offered  high-level  guidance  for 
SPI.  The  PCM  was  used  as  the  basis  for  the  plans  made  by  the  IATs,  and  for  support  to  them 
from  the  SEPG  PSG  West  wanted  to  use  existing  models  and  adapt  the  models  to  suit  their 
needs  rather  than  invent  approaches  and  strategies  from  scratch.  The  approach  to  using  these 
models  is  described  in  the  following  sections. 

1.4.1  The  IDEAL  Model 

The  IDEAL  model  consists  of  five  phases  of  improvement  in  an  organization:  Initiating,  Di¬ 
agnosing,  Establishing,  Acting,  and  Learning  (see  Figure  1)  [Gremba  1997,  McFeeley  1996]. 

5  In  1994,  early  in  the  relationship  between  Xerox  and  the  SEI,  an  initial  collaboration  led  to  the 
development  of  a  software  inspections  adoption  process  for  Xerox  based  on  one  SEI  consultant’s  prior 
experiences  [Ackerman  1983,  Fagan  1976],  Later,  an  adaptation  of  the  inspections  adoption  process 
for  application  to  software  technology  more  generally  was  developed  [Fowlerl996a,  Fowler  1996b],  in 
the  form  of  an  early  draft  of  the  Process  Change  Model  and  guidebook.  The  model  combined  aspects 
of  the  Xerox  Leadership  Through  Quality  problem-solving  process  with  the  generalized  adoption 
process,  and  the  guidebook  was  prepared  containing  guidance  on  the  application  of  the  model. 
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In  the  Initiating  phase,  an  organization  determines  the  need  for  change  due  to  business  condi¬ 
tions,  quality  issues,  or  other  fundamental  stresses.  In  the  Diagnosing  phase,  the  organization 
takes  stock  of  its  situation,  comparing  it  to  a  more  desirable  state.  This  phase  is  often  per¬ 
formed  as  an  audit,  perhaps  by  a  consulting  firm,  or  as  an  assessment,  such  as  those  per¬ 
formed  for  the  International  Standards  Organization  or  using  the  Software  CMM.  In  the  Es¬ 
tablishing  phase,  plans  are  laid  for  addressing  the  issues  identified  in  the  Diagnosing  phase; 
this  may  include  setting  up  teams  or  task  forces  to  take  action  on  the  issues.  In  the  Acting 
phase,  the  teams  execute  tasks  to  further  understand  issues  and  resolve  them  if  possible.  Fi¬ 
nally,  in  the  Learning  phase,  the  organization  looks  back  at  the  experiences  of  the  entire 
IDEAL  cycle,  and  notes  lessons  that  have  been  learned  and  steps  to  take  to  improve  their  per¬ 
formance  in  the  future. 

Figure  1:  The  IDEAL  model 


Learning 


Acting 
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1.4.2  The  Process  Change  Model:  Elaboration  on  IDEAL 

The  PCM  (see  Figure  2)  overlaps  and  expands  upon  the  Establishing,  Acting,  and  Learning 
phases  of  the  IDEAL  model,  and  is  focused  on  the  work  of  a  team  that  is  responsible  for  in¬ 
troducing  a  specific  software-related  change  (such  as  an  IAT).  One  or  more  teams,  depending 
on  the  size  of  the  organization  and  the  breadth  of  improvement-related  change  that  is  desired, 
follows  the  PCM  independently,  with  the  SEPG  performing  overall  coordination. 

Figure  2:  The  Process  Change  Model 


Activity  1 


There  are  eight  major  activities  in  the  PCM,  represented  at  their  most  abstract  level  in  Figure 
2.  In  Activity  1,  the  team — an  IAT,  in  the  case  of  PSG  West — is  established,  with  a  charter 
written  to  provide  scope,  establish  boundaries,  and  describe  team  member  roles.  During  Ac¬ 
tivity  2  through  Activity  4,  the  IAT  gathers  and  evaluates  the  requirements  for  the  particular 
area  it  will  focus  on;  in  PSG  West’s  case  the  requirements  were  derived  from  a  comparison 
between  the  KPA  being  introduced  and  the  baselined  current  state  of  KPA-related  work  in  the 
PSG  West  software  projects.  In  Activity  5,  the  IAT  develops  a  prototype  solution  based  on 
those  requirements,  possibly  in  conjunction  with  an  interested  project.  In  Activity  6  the  IAT 
performs  pilot  testing  of  the  prototype  solution,  preferably  on  more  than  one  project;  this  also 
allows  the  IAT  to  evaluate  the  process  of  adaptation  and  introduction  in  the  context  of  a  spe¬ 
cific  project.  Steps  in  Activity  7  address  “rolling  the  solution  out” — that  is,  adapting  and  de¬ 
ploying  a  solution,  project  by  project,  to  participants  in  the  particular  improvement  effort.  In 
Activity  8,  the  IAT  wraps  up  and  hands  off  materials  for  ongoing  support  to  a  central  group — 
the  SEPQ  in  the  case  of  PSG  West — and  to  other  functional  organizations  such  as  training, 
and  then  disbands. 
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1.4.3  Using  the  Models 

The  essence  of  the  use  of  the  PCM  and  IDEAL  models  was  that  the  PSG  West  SPI  effort  was 
managed  like  a  project,  with  the  SEPG  acting  as  project  manager.  The  SEPG  created  a  sched¬ 
ule  for  SPI  as  a  whole,  based  on  the  elements  in  the  IDEAL  model.  The  PCM  activities  pro¬ 
vided  the  basis  for  IAT-level  plans.  These  elements  were  tracked  and  reported  on  a  regular 
basis  during  the  SEPG  meetings  and  during  the  meetings  with  the  MST.  The  SEPG  used  a 
combination  of  the  two  models  to  provide  a  high  level  status  of  PSG  West’s  SPI  effort  (see 
Figure  3). 


Figure  3:  An  IDEAL  and  PCM-Based  Status  Report  Format 
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The  SEPG  took  the  lead  in  learning  about  Software  CMM,  SPI,  and  the  PCM.  Each  IAT  was 
assigned  a  member  of  the  SEPG  as  a  resource — a  consultant — in  these  areas.  This  allowed 
the  members  of  the  IAT  to  focus  on  the  specific  technical  or  process  work  related  to  their 
KPA. 

The  Requirements  Management  improvement  action  team  (RM  IAT)  and  the  combined  ac¬ 
tion  teams  for  Software  Project  Planning  and  Software  Project  Tracking  and  Oversight 
(SPP/SPTO  IAT)  used  the  PCM  directly,  with  lessons  passed  along  to  other  IATs  whose  work 
was  scheduled  to  take  place  later.  (For  these  latter  teams,  the  PCM  was  optional.)  Thus  the 
experiences  described  below  come  predominantly  from  those  two  IATs. 
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2  Activity  1 :  Establish  Team 


Each  section  in  this  and  the  following  chapters  describes  one  PCM  activity  and  its  purpose, 
lists  the  tasks  in  the  activity,  provides  highlights  of  work  done  by  one  or  more  IATs,  notes 
what  worked  well  and  what  problems  were  encountered,  and  ends  by  summarizing  lessons 
learned.  Some  issues  crossed  IAT  boundaries  and  affected  all  the  teams;  these  are  described 
here  as  SEPG-level  problems.  The  lessons  learned  that  are  listed  are  based  on  in-process  as 
well  as  post-process  feedback  from  the  IATs  using  the  PCM,  and  from  the  SEPG. 

2.1  Purpose 

The  purpose  of  Activity  1  was  to  set  up  an  IAT  and  develop  a  plan  for  improving  a  software 
development  project’s  process  in  the  area  of  a  specific  KPA.  Important  work  in  this  Activity 
included  confirming  sponsorship  by  a  specific  manager  for  the  IAT.  The  sponsoring  man¬ 
ager — “sponsor”  for  short — was  to  provide  administrative  and  strategic  guidance  to  the  IAT 
and  to  articulate  IAT  progress  or  issues  to  other  managers.  In  most  cases,  the  sponsor  was 
also  a  member  of  the  MST  and  a  project  manager  whose  project  would,  eventually,  partici¬ 
pate  in  a  pilot  implementation  of  an  IAT  solution.  Aligning  the  sponsor’s  perspective  with 
that  of  the  SEPG  consultant  supporting  the  particular  IAT,  as  well  as  with  the  IAT  leader  once 
he  or  she  was  appointed,  was  thus  particularly  important  in  the  early  life  of  an  IAT.  Other 
important  tasks  in  Activity  1  included  developing  an  IAT  charter,  appointing  IAT  members, 
holding  an  IAT  kickoff  session,  and  developing  an  IAT  plan.  Selecting  a  domain  expert — 
someone  who  had  experience  and  background  in  the  area  that  the  IAT  was  to  work  in — was 
also  included  in  Activity  1. 

2.2  Tasks 

1 .  Assign  SEPG  consultant  and  sponsor. 

2.  Draft  a  tactical  plan. 

3.  Align  the  understanding  of  the  sponsor  and  the  SEPG  consultant. 

4.  Select  the  IAT  leaders  and  members. 

5.  Conduct  a  management  awareness  workshop. 

6.  Finalize  the  tactical  plan  and  align  sponsor,  SEPG  consultant,  and  IAT  leader. 

7.  Conduct  a  “go/no  go”  meeting  for  IAT. 

8.  Conduct  the  IAT  initial  training. 
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9.  Conduct  the  IAT  kickoff. 

10.  Develop  an  improvement  activity  plan. 

1 1 .  Select  a  domain  expert. 

12.  Record  and  analyze  lessons  learned. 

2.3  Highlights  of  What  Was  Done 

Most  of  the  work  specified  in  Activity  1  was  already  underway  when  PSG  West  decided  to 
work  with  the  PCM.  Activity  1  PCM  tasks  were  revised  to  reflect  this  during  a  pre¬ 
collaboration  revision  process  which  customized  the  PCM  for  PSG  West  use.  PSG  West  kept 
to  its  plan,  developed  prior  to  the  collaboration,  to  initiate  one  IAT  about  every  six  weeks. 

This  approach  to  IAT  start-up  was  consistent  with  the  goal  to  make  the  RM  IAT  the  first  user 
of  the  PCM,  and  to  pass  along  lessons  from  its  early  use  to  the  SPP/SPTO  IAT  and  other  IATs 
interested  in  following  the  PCM. 

2.3.1  Establishing  the  Management  Focus  Teams 

Early  in  the  SPI  start  up  activities,  three  management  focus  teams  (MFTs)  were  established 
by  the  MST  to  address  the  four  Software  CMM  common  features,  which  include  common 
elements  that  support  the  activities  performed  by  a  KPA,  such  as  policy  statements,  sponsor¬ 
ship,  allocation  of  resources  and  funds,  and  training.  The  size  of  the  MFTs  ranged  from  three 
to  five  members.  The  PCM  did  not  address  establishing  the  MFTs  explicitly,  although  its 
“whole  product”  approach  [Moore  1991]  implied  preparing  materials  and  taking  actions  re¬ 
garding  common  features  and  other  support  for  implementing  KPA  activities. 

2.3.2  Establishing  the  Improvement  Action  Teams 

The  MST  chartered  five  IATs  to  create  solutions  to  the  problems  identified  in  the  CBA IPI. 
Software  engineering  professionals  drawn  from  PSG  West  projects  staffed  each  IAT;  one  of 
the  members  was  appointed  to  be  the  team  leader.  Each  IAT  was  tasked  to  improve  and/or 
implement  software  practices  in  a  specific  KPA. 

IATs  met  weekly  for  two  to  four  hours,  and  worked  outside  the  meetings  for  about  another  six 
hours,  with  an  occasional  full-day  meeting  off  site.  The  size  of  the  IATs  ranged  from  6  to  10 
members.  The  teams  followed  the  activities  in  the  PCM,  coached  by  the  SEPG  member  as¬ 
signed  to  their  team  as  consultant.  Their  goal  was  to  prepare  KPA-specific  process  descrip¬ 
tions  and  related  materials  that  would  help  software  projects  adopt,  maintain,  and  evolve  the 
processes,  and  plan  their  deployment. 

2.3.3  Training  Members  of  the  Improvement  Action  Team  in 
SPI-Related  Concepts 

The  SEPG  was  responsible  for  facilitating  the  development  and  progress  of  the  IATs,  including 
Software  CMM-  and  SPI-related  training.  Xerox  did  not  have  internal  training  courses  related  to 
SPI  at  that  time,  so  the  SEPG  provided  some  training  itself,  developing  the  materials  based  upon 
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its  own  training.  Other  training,  in  KPA-specific  topics,  was  acquired  as  needed  by  the  IATs.  In¬ 
ternally  available  courses  related  to  team  and  quality  methods  were  also  used.  Training  subjects, 
audiences,  and  sources  of  material  and/or  instructions  are  listed  in  Table  1.  (For  completeness,  we 
list  all  courses  here,  even  though  several  were  not  used  until  later  PCM  activities.) 


Table  1:  Training  Received  by  Improvement  Action  Teams 


IAT  Training  Topic 

Audience 

Source  of  Material/Instruction 

Software  CMM  overview 

All  IAT  and  MFT  members, 

MST  members,  first-line  man¬ 
agers,  and  project  leaders 

Software  CMM  [Paulk  1991] 
and  PSG  West  SEPG  training 

Overview  of  PSG  West  SPI 

strategic  plan  and  SPI  infra¬ 
structure 

All  IAT  and  MFT  members, 

MST  members,  first-line  man¬ 
agers,  and  project  leaders 

PSG  West  SPI  strategic 
plan/PSG  West  SEPG 

Overviews  of  specific  KPAs 

All  IAT  and  MFT  members 

Software  CMM  [Paulk  1991] 

and  PSG  West  SEPG 

Requirements,  and  a  software 

tool  for  RM 

RM  IAT  members 

Vendor 

Team  methods 

RM  IAT  members 

Xerox  internal  training 

Root  cause  analysis 

Software  configuration  man¬ 
agement  (SCM)  IAT  members 

Xerox  internal  training 

Goal/question/metric  technique 

Measurement  and  analysis  MFT  i 

i 

members 

Vendor 

Software  project  planning  and 
management 

SPP/SPTO  IAT  members; 

SEPG  and  MST  members;  man¬ 
agers  involved  in  the  pilot  use 

of  the  SPP/PTO  solution 

Vendor 

Software  quality  assurance 

SQA  and  SEPG  members 

Vendor 
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2.3.4  The  Lead  Goose  Strategy  for  Launching  Improvement 
Action  Teams 

The  PSG  West  SEPG  decided  to  stagger  the  initiation  of  IATs  (and  MFTs),  based  on  the  expe¬ 
riences  with  SPI  in  other  Xerox  organizations.  (See  Figure  4.)  This  “lead  goose”  strategy 
meant  that  the  earlier  teams  might  take  longer  to  get  going,  and  might  have  to  do  more  work 
to  develop  the  first  instances  of  materials  and  tactics,  but  the  net  effort  of  all  the  teams  would 
be  reduced.  In  effect,  the  first  IAT  would  break  the  trail  for  the  other  IATs,  reducing  their  risk 
by  passing  lessons  back  to  them  as  they  started,  and  by  developing  materials  such  as  tem¬ 
plates  that  could  be  reused.  The  SEPG  members  who  were  waiting  for  their  IATs  to  begin 
work  would  help  the  lead  IATs  to  prepare  these  materials. 


Figure  4:  The  SPI  Activity  Start-Up  Time  Line:  IAT  and  MFT  Start-Up  Schedule 
Showing  the  RM  IAT  as  “Lead  Goose” 
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2.4  What  Worked  Well 

2.4.1  Membership 

Overall,  Activity  1  was  a  positive  experience  for  the  IATs.  Most  IATs  felt  that  strong  mem¬ 
bers  had  been  selected:  they  had  the  needed  experience,  skills,  and  expertise;  worked  well 
together;  and  supported  the  work  they  were  assigned  to  do.  Attrition  was  occasionally  a 
problem:  one  team  even  lost  a  member  before  team  kickoff.  The  consensus  was  that  IATs 
needed  to  be  staffed  in  anticipation  of  losing  members  due  to  relocation,  promotion,  resigna¬ 
tion,  etc.  Some  team  members  observed  that  the  IAT  served  as  a  “good  networking  forum.” 

2.4.2  Support  from  SEPG 

The  PCM  suggested  using  examples,  checklists  and  templates,  but  provided  only  a  few  of 
these.  And  in  fact,  the  creation  of  document  templates  was  well  underway  at  PSG  West  prior 
to  the  start  of  the  SEI/Xerox  collaboration,  and  was  one  of  the  most  useful  activities  in  the 
direct  support  of  IATs  by  the  SEPG.  The  SEPG  created  templates  to  help  IATs  facilitate  rapid 
development  of  materials  and  to  provide  a  standardized  approach.  Then  once  one  IAT  had 
used  a  template,  an  example  plan  was  also  available.  For  example,  a  template  for  the  piloting 
plan  was  developed,  used  by  the  RM  IAT  to  create  a  piloting  plan,  and  then  reused  by  other 
IATs  subsequently,  with  each  sharing  an  example  of  its  plan. 

2.5  lAT-Level  Issues 

2.5.1  PCM  Training 

The  IATs  felt  that  training  on  the  PCM  was  too  limited,  as  the  SEPG  consultant  for  each  IAT 
had  provided  only  a  brief  overview  at  the  kickoff  of  each  team,  with  the  assumption  that  de¬ 
tailed  guidance  would  come  through  the  consulting  relationship.  As  IATs  began  to  follow  the 
guidance  in  the  PCM  in  detail,  it  was  not  clear  to  them  how  the  individual  tasks  supported  the 
overall  objective  of  the  PCM  process — to  create  usable  solutions  and  pilot  them.  IATs  stated 
that  they  needed  more  of  a  “big-picture”  perspective  from  the  SEPG  and  the  SEI  coach. 


2.5.2  PCM  Value 

The  IATs  were  unclear  about  the  value  of  the  PCM.  Also,  one  team  expressed  specific  con¬ 
cerns  about  the  PCM  style  and  format;  the  consistency  of  its  terminology  with  that  of  other 
SEI  sources;  an  uneven  level  of  detail;  and  inconsistencies  between  sections. 

2.5.3  Level  of  IAT  Effort 

Regarding  level  of  effort  estimated  for  IAT  work,  some  teams  reflected  that  10  hours  per 
week  might  not  be  enough  time  to  do  the  job  they  were  given  and  that  it  was  difficult  to  co¬ 
ordinate  all  of  the  team  members’  schedules. 
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2.6  SEPG-Level  Issues 

2.6.1  Extent  of  IAT  Training 

A  general  concern  was  how  much  training  IAT  members  should  receive,  and  whether  training 
from  the  SEPG  was  enough.  Some  IATs  wanted  training  directly  from  the  SEI  or  from  ven¬ 
dors.  There  was  consensus  that  the  SEPG  needed  to  better  understand  what  members  of  an 
IAT  knew  prior  to  training.  Conveying  the  “big  picture”  of  the  PCM  was  particularly  chal¬ 
lenging,  because  the  SEPG  itself  had  had  training  on  the  PCM  but  did  not  have  experience 
using  it. 

2.6.2  Managing  the  Collaboration  Day  to  Day 

The  relationship  between  the  PSG  West  SEPG  and  the  SEI  was  designed  to  filter  the  interac¬ 
tion  between  the  SEI  coach  and  the  IATs.  This  had  the  positive  effect  of  channeling  concerns 
through  the  SEPG  and  allowing  for  organizational  learning — if  all  SEPG  members  knew  of 
lessons  and  issues,  then  all  IAT  members  could  be  assured  of  access  to  these.  It  also  had  a 
negative  effect  by  delaying  the  SEI  coach’s  awareness  of  concerns  as  well  as  her  ability  to 
provide  clarification  to  both  the  SEPG  and  the  IAT. 

2.7  Lessons  Learned 

•  Anticipate  attrition  when  staffing  IATs. 

It  is  inevitable  that  IAT  members  will  be  lost.  The  team  needs  to  be  large  enough  so  that 
when  there  is  turnover  in  membership,  a  core  group  that  can  maintain  momentum  still  re¬ 
mains.  Focusing  on  IAT  leader  selection  is  necessary  but  not  sufficient  in  the  early  life  of  an 
IAT. 

•  Identify  back-up  sponsors. 

There  may  also  be  attrition  in  the  group  of  sponsoring  managers.  Some  managers  may  be 
more  interested  in  SPI  than  others  or  may  have  more  time  available.  Early  identification  of 
additional  sponsor  candidates  allows  for  including  them  in  SPI-related  activities  from  the 
start.  A  group  of  managers  educated  about  and  interested  in  SPI  is  then  available  to  draw 
from  as  needed. 

•  Overprepare  SEPG  members  for  critical  presentations. 

If  SEPG  members  are  using  a  common  process  (such  as  the  PCM)  for  IAT  work,  rehearse  any 
process  overview  presentation  given  by  SEPG  members  with  the  process  coach  (in  this  case 
the  SEI  coach)  and/or  each  other.  This  allows  consolidation  of  SEPG  knowledge  of  the  proc¬ 
ess  and  a  smoother  presentation  to  IATs.  As  experience  is  gained,  SEPG  consultants  working 
with  IATs  kicking  off  later  will  then  be  able  to  get  better  support  from  those  working  with 
earlier  IATs. 
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•  Complete  document  templates  prior  to  starting  lATs. 

The  PCM  focused  on  creation  of  templates  by  the  IATs  for  the  eventual  users  of  the  solutions 
created  by  the  IATs.  It  is  equally  important  to  prepare  templates  for  the  IATs  to  use,  and  these 
need  to  be  ready  prior  to  the  IATs’  beginning  to  use  the  PCM  or  other  common  process 
model. 

•  Minimize  the  adjustment  required  of  teams  when  a  process  is  new  or  rough. 

When  using  a  prototype  document  such  as  the  PCM,  make  sure  it  looks  as  much  as  possible 
like  a  finished  product.  Provide  formatting  and  terminology  that  is  consistent  with  docu¬ 
ments  that  IAT  members  are  accustomed  to  working  with.6 


6  This  lesson  was  applied  by  the  IATs  as  they  prepared  their  process  solutions,  and  expedited  solution 
acceptance  by  both  the  MST  and  the  adopting  projects. 
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3  Activity  2:  Define  Desired  Process 


3.1  Purpose 

The  purpose  of  Activity  2  was  to  create  a  requirements  specification  for  the  process  a  project 
would  follow  to  be  consistent  with  a  particular  KPA.  The  IAT’s  task  was  to  determine  what 
was  needed  for  that  KPA  process  for  projects  at  PSG  West.  The  simplest  approach  to  this  was 
to  use  the  Software  CMM  itself;  another  approach  was  to  use  the  Software  Process  Frame¬ 
work  (SPF)  [Olson  1994],  which  restates  the  Software  CMM  in  a  process  format  and  can 
serve  as  a  starting  point  for  process  definition. 

The  essence  of  this  step  in  the  PCM  was  interpreting  the  Software  CMM  so  that  its  applica¬ 
bility  to  PSG  West  projects  was  clear.  This  meant  determining  what  outputs  and  states  would 
result  from  a  project’s  use  of  a  Level  2  KPA  process;  who  received  those  outputs  or  was  af¬ 
fected  by  the  state  change;  how  the  Software  CMM  statements  might  be  translated  into  a  pro¬ 
cess  format  that  PSG  West  projects  would  understand;  how  current  policy  might  need  to  be 
revised  to  support  the  envisioned  new  process  and  its  outcomes;  and  how  the  process  would 
be  delivered  (what  training,  documentation,  and  behavior  changes  would  allow  a  project  to 
execute  that  process). 

3.2  Tasks 

1 .  Identify  the  KPA  process  output. 

2.  Identify  the  customer  (for  the  output). 

3.  Identify  the  KPA  process  requirements. 

4.  Translate  the  KPA  process  requirements  into  specification. 

5.  Review  the  draft  policy  (for  the  KPA  process). 

6.  Identify  the  initial  whole  product  “wheel”  (what  things  besides  the  process  would  be 
needed  to  help  users  use  the  process) . 

7.  Record  and  analyze  lessons  learned. 

3.3  Highlights  of  What  Was  Done 

3.3.1  Educating  the  Team 

The  RM IAT  located  and  viewed  a  tape  from  the  SEI  Software  Engineering  Series  on  re¬ 
quirements  management  [Zelesnik  1992],  The  RM  IAT  also  looked  at  the  Software  Process 
Framework  for  an  example  of  how  requirements  management  looked  when  specified  as  a 
process.  Later,  the  team  brought  in  a  vendor  to  obtain  training  on  requirements  engineering  as 
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well  as  on  a  related  software  tool,  acknowledging  that  the  RM  IAT  would  have  to  select  a 
subset  of  material  from  what  they  learned  in  order  to  address  RM  as  a  KPA.  The  SPP/SPTO 
IAT  brought  in  training  on  project  planning  from  another  vendor;  this  proved  very  valuable, 
particularly  in  the  area  of  estimation.  Other  IATs  brought  in  courses  on  software  measure¬ 
ment  and  quality  assurance.  Again,  these  were  not  tailored  to  implementation  of  particular 
KPAs. 

3.3.2  Gathering  Process  Requirements 

The  RM  IAT  knew  that  it  could  not  create  processes  without  adapting  them  to  suit  local  con¬ 
ditions.  Thus  the  RM  IAT  organized  a  “roundtable”  meeting,  inviting  marketing,  planning, 
and  software  development  representatives  from  each  project  participating  in  the  SPI  effort. 
The  purpose  of  the  meeting  was  to  solicit  input  on  how  best  to  manage  requirements  from  the 
perspective  of  people  in  various  roles.  Participants  were  sent  the  draft  requirements  manage¬ 
ment  policy,  the  draft  requirements  specification  for  the  requirements  management  KPA,  and 
questions  to  stimulate  thinking.  The  SPP/SPTO  IAT  built  on  the  RM  IATs  lessons  learned, 
using  the  same  approach  but  with  changes.  Its  roundtable  was  made  a  two-stage  process, 
where  the  first  meeting  was  orientation  and  the  second  meeting  focused  on  gathering  data. 

3.4  What  Worked  Well 

The  roundtable  format  first  used  by  the  RM  IAT  was  successful,  and  was  also  used  by  other 
IATs.  In  gathering  process  requirements,  members  of  the  RM  IAT  needed  to  discuss  and  con-, 
cur  on  a  common  interpretation  of  the  Software  CMM  practices  for  requirements  manage¬ 
ment.  They  clarified  who  the  “customer”  was  for  the  RM  process,  with  a  list  of  specific  cus¬ 
tomers  to  refer  back  to,  and  built  a  common  understanding  of  RM  for  PSG  West.  They  used 
the  SPF  to  understand  how  RM  looked  as  a  process. 

3.5  lAT-Level  Issues 

3.5.1  IAT  Level  of  Effort 

Based  on  records  kept  by  the  SEPG,  it  appears  that  people  were  not  able  to  devote  as  much 
time  to  the  IATs  as  originally  planned.  Based  on  a  basis  of  10  hours  per  week  per  IAT  mem¬ 
ber,  240  hours  of  IAT  work  were  planned  in  the  first  month  of  RM  IAT  operation,  and  200 
hours  in  the  second  month.  Actual  hours  were  79  and  80,  respectively.  Project  work  commit¬ 
ments  conflicted  with  members ’getting  to  IAT  meetings.  The  PCM  did  not  suggest  a  particu¬ 
lar  approach  to  IAT  effort  or  frequency  of  meetings;  at  Xerox,  teams  were  set  up  to  work  on 
quality  and  other  corporate  issues,  and  these  typically  met  every  week,  so  the  IATs  were 
structured  in  the  same  way. 

3.5.2  The  PCM  and  IATs  as  Change  Agents 

As  the  collaboration  between  PSG  West  and  the  SEI  continued,  it  became  clear  that  the  al¬ 
ready  steep  learning  curve  for  the  Software  CMM  and  software  process  improvement  was 
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made  steeper  for  the  IATs  as  they  also  tried  to  leam  both  about  their  KPA-related  change  and 
about  being  change  agents  according  to  the  PCM. 

3.5.3  Software  CMM-Specific  Training  for  IATs 

Very  little  training  and  education  specific  to  requirements  management  or  other  KPAs  was 
available  in  1995.  While  the  Level  2  KPAs  were  well  known  areas  within  software  engineer¬ 
ing  generally,  their  specific  presentation  and  definition  in  the  Software  CMM  was  not  yet 
widely  addressed  by  Xerox  internal  training,  commercial  training,  or  consulting  organiza¬ 
tions.  For  example,  it  was  fairly  easy  to  find  a  course  on  requirements  engineering  or  systems 
engineering,  or  software  project  management,  but  it  was  difficult  to  find  courses  that  ad¬ 
dressed  requirements  management  or  project  management  as  defined  in  the  Software  CMM. 
And  while  LAT  members  had  had  experience  in  the  particular  areas  where  they  were  working, 
they  had  not  had  experience  framing  those  areas  according  to  the  Software  CMM;  much  edu¬ 
cation,  both  from  outside  and  from  their  own  personal  and  group  efforts,  was  needed.  This 
had  not  been  clearly  spelled  out  in  the  PCM,  in  part  because  the  PCM  was  not  Software 
CMM-specific. 

3.5.4  KPA  Process  Requirements:  The  Roundtable  and  the 
SPF 

Attendees  at  the  first  roundtable  meeting  held  by  the  RM  LAT  did  not  immediately  distinguish 
between  the  “requirements”  referred  to  by  requirements  management,  and  the  requirements 
for  a  process  to  do  work  in  a  KPA  —  in  this  case  the  requirements  management  process. 
Nonetheless,  after  this  was  clarified,  the  team  felt  that  the  roundtable  was  a  success  and  re¬ 
ceived  good  information  from  it. 

The  RM  team’s  rewrite  of  the  requirements  management  process  of  the  SPF  to  adapt  it  for 
PSG  West  took  longer  than  desired;  the  detail  in  the  SPF  was  a  distraction  for  the  team  mem¬ 
bers,  who  were  new  to  developing  process  descriptions.  In  addition,  the  team  discovered  that 
requirements  terms  obtained  from  the  roundtable  discussion  and  other  investigations  were  not 
used  consistently  across  PSG  West  projects  and  that  the  current  approach  to  requirements 
biased  the  terminology  that  they  wanted  to  use  for  the  revised  PSG  West  process. 

As  the  RM  and  SPP/SPTO  IATs  gained  experience,  useful  lessons  were  passed  to  the  IATs 
that  had  started  later.  For  example,  in  the  case  of  defining  the  desired  state  in  Activity  2,  one 
member  said,  “Don’t  get  trapped  defining  what  your  project  does  now — describe  what  the 
new  process  will  do.”7  Another  member  said,  “Draft  a  process — not  the  procedures  or  how  to 
do  it,  but  what  to  do.” 


7  Much  later  in  the  collaboration,  PSG  West  IATs  indicated  that  they  felt  they  were  required  to  start 
with  a  “clean  slate”  in  developing  process  solutions.  This  perception  caused  them  to  assume  that  they 
could  not  use  current  good  practices  from  PSG  West  projects,  and  definitely  slowed  down  both  the 
solution  development  and  solution  adoption  efforts. 
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The  PCM  focused  on  getting  the  requirements  and  then  developing,  in  conjunction  with 
stakeholders,  a  vision  of  how  those  requirements  might  be  transformed  into  a  process  for  RM 
or  one  of  the  other  KPAs.  Guidance  on  how  to  elicit  the  requirements  was  not  provided, 
based  on  the  assumption  that  software  project  members  would  have  experience  with  require¬ 
ments.  In  addition,  the  PCM  did  not  make  clear  what  level  of  detail  was  required,  or  how  the 
current  process  in  a  software  project  might  feed  into  the  requirements-gathering  process. 

3.6  SEPG-Level  Issues 

3.6.1  Providing  KPA-Specific  Knowledge  to  lATs:  Training 

Once  Activity  2  was  underway,  IAT  members  immediately  saw  the  need  for  more  detailed 
knowledge  as  they  worked  to  create  a  specific  vision  for  how  each  KPA  process  would  work 
at  PSG  West.  This  problem  was  handled  at  the  IAT  level,  but  it  might  have  been  handled 
more  effectively  at  the  SEPG  level,  had  it  been  recognized  earlier  in  the  SPI  planning  proc¬ 
ess.  The  SEPG  and  LATs  acknowledged,  during  Activity  2  and  later  in  Activity  5,  that  more 
detailed  training  and/or  education  in  their  respective  KPAs  should  have  occurred  during  Ac¬ 
tivity  1,  in  addition  to  the  more  general  courses  that  were  provided. 

3.6.2  Providing  KPA-Specific  Knowledge  to  lATs:  Domain 
Expert 

The  PCM  suggested  that,  in  addition  to  KPA-specific  training  for  the  LATs  in  Activity  1 ,  a 
domain  expert  be  selected  to  serve  as  an  adjunct  member  of  the  IAT.  A  domain  expert,  as  de¬ 
scribed  in  the  PCM,  would  have  experience  eliciting  requirements  from  stakeholders  and 
adapting  existing  approaches  to  a  KPA,  to  develop  the  “desired  process”  that  is  the  objective 
of  Activity  2.  Some  issues  in  Activity  2  came  about  as  a  result  of  not  having  domain  experts 
in  place  for  the  IATs. 

3.7  Lessons  Learned 

•  Be  sure  the  IAT  has  content  expertise. 

Even  if  team  members  believe  that  they  share  a  common  understanding  of  a  KPA,  viewing  a 
video  or  taking  a  one-  or  two-day  class  as  a  group  early  in  the  life  of  the  team  ensures  that  all 
members  have  been  exposed  to  the  same  terminology.  The  IAT  should  get  expert  help  as  soon 
as  possible,  especially  for  learning  about  how  other  organizations  have  approached  KPA  so¬ 
lution  development.  Having  access  to  a  domain  expert  can  compensate  for  incomplete 
knowledge  early  in  the  life  of  an  IAT. 

•  Hold  a  roundtable  meeting  to  get  customer  input.  Provide  Software  CMM  and  SPI 
orientation. 

Use  a  two-stage  approach  to  gathering  process  requirements  from  projects,  with  the  first 
stage  being  orientation  to  the  Software  CMM  and  SPI  (or  other  frameworks)  and  the  second 
being  data  gathering.  A  major  lesson  learned  was  that  many  attendees  didn’t  know  about  SPI 
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or  the  Software  CMM,  so  the  roundtable  took  longer  than  expected,  and  a  second  meeting 
had  to  be  scheduled  to  finish  the  work. 

•  Use  packaged  processes  with  caution . 

Packaged  processes  such  as  those  in  the  SPF  or  purchased  from  vendors  can  save  time,  but 
only  if  not  followed  literally.  Examining  several  packaged  processes  might  clarify  what  can 
be  safely  simplified  or  left  out.  It  is  best  to  look  at  processes  that  have  been  used  in  the  same 
application  areas  as  those  in  which  your  organization  works. 

•  Strike  a  balance  between  drawing  out  the  best  of  what  is  and  the  best  of  what  can  be  with 
the  desired  process . 

Build  on  the  best  of  your  current  process  while  remaining  open  to  new  approaches.  No  single 
project  may  have  all  the  answers,  but  together  good  practices  from  a  number  of  projects  may 
provide  an  excellent  solution.  Use  external  materials  to  enhance  this  composite  solution,  re¬ 
membering  that  the  solution  in  hand  is  already  one  with  which  project  personnel  are  familiar. 
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4  Activity  3:  Establish  Current  State 


4.1  Purpose 

The  purpose  of  Activity  3  was  to  gather  enough  data  to  characterize  the  current  state  of  prac¬ 
tice  in  the  KPA  being  improved  within  PSG  West  projects.  This  activity  required  meetings  of 
the  SEPG  and  IAT  leaders  with  the  MST  to  determine  which  projects  would  be  baselined,  and 
then  a  review  and  analysis  of  existing  processes  in  those  projects.  One  objective  was  to  note 
where  good  practices  and  policies  were  already  in  place  so  that  these  could  be  built  upon  in 
developing  solutions.  The  other  objective  was  a  compilation  of  a  baseline  of  practices  com¬ 
mon  across  the  projects.  (While  a  CBA-IPI  produces  similar  information,  under  the  ground 
rules  for  an  assessment,  this  information  must  be  kept  confidential.)  Having  a  detailed  base¬ 
line  allowed  the  LATs  to  determine  the  difference  between  what  worked  well  and  what  needed 
to  be  changed,  and  to  use  that  as  the  basis  for  defining  KPA-based  processes  at  PSG  West. 

4.2  Tasks 

1 .  Identify  the  projects  to  be  baselined. 

2.  Review  and  analyze  existing  processes. 

3.  Baseline  common  practices  across  all  the  projects  participating  in  SPI  or  other 
improvement  efforts. 

4.  Record  and  analyze  lessons  learned. 

4.3  Highlights  of  What  Was  Done 

4.3.1  Identifying  Projects  to  Baseline 

The  SEPG  and  MST  initially  identified  projects  that  would  participate  in  the  SPI  effort. 

These  projects  were  to  be  baselined  and  become  pilot  sites  for  the  solutions  developed  by  the 
LATs.  (Later,  when  specific  LATs  were  looking  for  pilot  sites,  the  appropriateness  of  a  project 
would  be  re-examined.  If,  for  example,  the  project  was  well  past  the  requirements  stage,  then 
it  would  not  be  suitable  for  a  pilot  of  the  RM  LATs  solution.) 

4.3.2  Interviewing  Members  of  Baseline  Projects 

The  RM  IAT  developed  questions  based  on  the  Software  CMM  to  use  when  interviewing 
members  of  projects  that  would  be  baselined.  The  idea  behind  the  interviews  was  to  deter¬ 
mine  the  current  practice  in  the  projects.  Interviewers  included  one  person  on  the  RM  IAT 
who  was  also  a  project  member  and  another  RM  IAT  member  who  was  not  a  member  of  the 
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project  being  interviewed.  Interviewing  took  slightly  more  than  a  week  of  calendar  time  to 
complete.  The  primary  person  from  a  project  to  be  interviewed  was  the  program  manager. 

Baseline  metrics  were  defined  so  that  the  RM IAT  had  a  way  to  characterize  what  particular 
projects  were  doing  in  requirements  management.  The  process  specification  for  requirements 
management  developed  in  Activity  2  was  used  as  a  checklist  during  the  interviews. 

The  objective  of  the  “lead  goose[cdi]”  strategy  was  that  one  IAT  would  try  a  task  and  pass 
along  the  lessons  it  learned  from  the  attempt.  In  this  case,  the  other  IATs  and  the  measure¬ 
ment  and  analysis  MFT  followed  the  example  of  the  second  IAT  performing  baselining  inter¬ 
views  (the  SPP/SPTO  IAT),  perhaps  due  to  the  RM  IATs  experience  of  the  time-consuming 
nature  of  using  the  SPF.  The  SPP/SPTO  IAT  developed  interview  questions  without  refer¬ 
ence  to  the  Software  CMM  or  the  SPF. 

4.4  What  Worked  Well 

In  general,  the  IATs  handled  Activity  3  well.  Most  had  experience  preparing  questionnaires 
and  doing  interviews,  and  this  paid  off.  The  teams  used  a  variety  of  strategies.  The  RM  IAT 
felt  that  having  a  checklist  helped  to  structure  their  baselining  interviews  and  to  perform  them 
consistently  across  a  wide  range  of  projects.  The  SPP/SPTO  IAT  adjusted  the  format  of  the 
interview  questions  so  that  terms  were  defined  in  a  statement  placed  immediately  beneath  the 
question  being  considered.  This  IAT  also  allowed  the  interviewees  to  keep  the  questions. 
Having  an  IAT  interviewer  with  some  product  knowledge  helped  in  communicating  with  the 
interviewees.  The  SCM  IAT  sent  interview  questionnaires  to  interviewees  ahead  of  time  so 
that  they  could  be  completed  before  the  actual  interview.  People  were  eager  to  participate  in 
the  interviews.  The  process  was  helpful  to  the  SCM  IAT;  team  members  met  and  began  to 
build  relationships  with  their  customers  for  the  SCM  process  that  the  IAT  would  develop. 

4.5  SEPG-Level  Issues 

The  IATs  reported  to  the  SEPG  that  interviewees  from  the  projects  being  baselined  for  SPI 
expressed  the  need  to  know  where  to  focus  when  multiple  initiatives  required  their  attention. 
This  was  due  in  part  to  another  major  organizational  change  effort  that  was  under  way  at  the 
same  time  as  the  SPI  effort.  The  SEPG  worked  to  link  and  coordinate  with  the  change  related 
to  this  other  effort,  and  to  respond  to  the  suggestion  of  the  IATs  that  this  should  be  addressed 
as  part  of  multi-level  communications  about  SPI.  (The  PCM  did  not  address  organizational 
change  unrelated  to  technology  change.) 

4.6  Lessons  Learned 

•  For  baselining  projects ,  use  a  matrix  based  directly  on  the  Software  CMM . 

Use  of  a  Software  CMM-based  matrix  rather  than  one  based  on  the  process  specification  for 
requirements  management  from  Activity  2  would  have  allowed  some  concurrent  execution  of 
tasks  in  Activity  2  and  Activity  3.  The  statements  in  the  Software  CMM  are  in  effect  a  set  of 
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requirements  for  complying  with  a  particular  KPA,  and  so  can  be  used  as  questions  to  deter 
mine  if,  for  example,  projects  are  performing  a  particular  activity  or  have  a  certain  policy  in 
place.  Using  the  Software  CMM  and  SPF  as  the  basis  for  developing  interview  questions 
saved  considerable  time.  Suggestions  for  how  to  develop  these  questions  and  provision  of 
examples  would  help  expedite  the  baselining  process. 

•  Send  questions  to  interviewees  ahead  of  the  interview. 

In  the  case  of  PSG  West,  this  gave  interviewees  a  chance  to  become  familiar  with  the  mate¬ 
rial,  and  the  interviews  went  more  smoothly.  Including  definitions  of  terminology  was  im¬ 
portant,  and  especially  helpful  when  these  definitions  were  next  to  the  question  where  the 
terms  were  used.  Interviewees  appreciated  being  able  to  keep  a  copy  of  the  questions. 
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5  Activity  4:  Identify  Gap 


5.1  Purpose 

The  purpose  of  Activity  4  was  to  compare  the  findings  from  Activity  3  on  the  current  state  in 
PSG  West  projects,  with  the  desired  state  information  gathered  in  Activity  2.  The  gap  be¬ 
tween  the  two  served  as  a  basis  for  estimating  how  much  work  was  needed  to  achieve  the 
desired  state  of  successful  implementation  and  the  use  of  process  solutions  for  Software 
CMM  Level  2  KPAs.  Issues  to  be  examined  included  systemic  sources  of  this  gap  (root 
causes),  effort  required  to  address  these,  and  related  risks.  After  this  activity  was  completed, 
an  IAT  would  have  a  strong  understanding  of  the  difficulty  both  of  developing  its  task  and  of 
deploying  the  product  solution.  This  would  then  serve  as  the  basis  for  renegotiating  its  charter 
and  plan  with  sponsors,  and  revising  either  as  needed.  The  techniques  for  performing  this 
activity  were  drawn  from  quality  methods  [Brassard  1994]. 

5.2  Tasks 

1 .  Analyze  organizational  risk  associated  with  change  by  performing  “fishbone”  analysis 
of  problem  symptoms  [Ishikawa  1982]. 

2.  Identify  problem. 

3.  Analyze  problem.  (Determine  root  cause.) 

4.  List  organization  risks  and  mitigation  strategies. 

5.  Estimate  time  required  for  change  and  magnitude  of  change. 

6.  Record  and  analyze  lessons  learned. 

5.3  Highlights  of  What  Was  Done 

5.3.1  Risk  Analysis  for  Organizational  Change 

The  RM  IAT  performed  the  first  task  in  Activity  4,  but  decided  to  skip  the  remaining  tasks 
except  for  the  lessons  learned  analysis.  The  team  used  a  hierarchy  of  organizational  change, 
reprised  in  the  PCM,  that  ranged  from  change  affecting  skills  and  procedures  to  change  re¬ 
quiring  adjustments  in  structure,  strategy,  and  culture  [Adler  1990]  (see  Figure  5).  The  team 
identified  problems  related  to  implementing  requirements  management;  these  fell  into  each  of 
the  categories — each  “bone”  on  the  fishbone  chart — of  the  hierarchy. 

The  RM  IAT  later  did  informal  root  cause  analysis  which,  in  turn,  led  to  the  development  of 
mitigation  strategies  for  use  later  in  the  solution  development. 
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Figure  5:  Hierarchy  of  Types  of  Organizations  from  Adler  and  Shenhar 


Level  of  Learn  ing  Required 


5.4  What  Worked  Well 

Despite  initial  difficulties  in  understanding  the  steps  in  Activity  4,  most  IATs  indicated  that 
gap  analysis  work  led  to  an  improved  understanding  of  the  risks  they  were  facing  in  devel¬ 
oping  and  implementing  solutions.  In  the  case  of  the  RM  IAT,  they  were  able  to  create  more 
cogent  statements  of  the  risks  in  implementing  a  requirements  management  process  solution 
at  PSG  West. 

The  SPP/SPTO  IAT  felt  that  the  results  of  their  root  cause  analysis  work  were  excellent.  The 
IAT  tracked  its  time  and  determined  that  each  of  the  10  root  causes  identified  took  about  1.5 
hours  to  find.  In  addition,  as  part  of  reviewing  Activity  4  prior  to  starting  it,  this  team  chose 
to  be  selective  about  PCM  tasks,  interpreting  them  as  it  supported  their  objectives. 

The  SCM  IAT  brought  in  a  root  cause  analysis  trainer  from  Xerox  corporate  training,  who 
developed  a  training  session  customized  to  their  needs.  The  IAT  also  narrowed  its  scope  prior 
to  beginning  the  root  cause  analysis.  This  analysis  was  considered  essential  in  identifying  the 
core  issues  by  the  SCM  IAT.  This  team  also  acknowledged  being  able  to  build  on  the  IATs’ 
work  that  preceded  the  SPI  work. 

5.5  lAT-Level  Issues 

The  RM  IAT  felt  that  the  PCM  wasn’t  clear  in  specifying  how  the  tasks  in  Activity  4  led  to 
later  activities  in  the  PCM.  Their  understanding  of  root  cause  analysis  from  prior  experience 
or  training  at  Xerox  was  that  it  was  a  lengthy,  detailed  process.  When  the  SEI  coach  ex¬ 
plained  that  the  PCM  approach  to  root  cause  was  an  informal  process  for  quickly  getting  at 
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the  essence  of  issues,  the  RM IAT  expedited  development  of  risk  statements  and  mitigation 
strategies. 

The  SPP/SPTO  IAT  initially  relied  heavily  on  the  PCM,  and  expressed  concern  when  it  did 
not  provide  enough  specific  guidance  or  when  it  was  inconsistent.  They  also  stated  that  they 
felt  like  “guinea  pigs”  because  the  PCM  was  being  changed  too  frequently  in  response  to 
their  problems  with  it. 

5.6  SEPG-Level  Issues 

The  experiences  of  the  RM  IAT  indicated  that  the  descriptions  of  tasks  in  Activity  4  needed 
clarification  and  lacked  consistency  with  other  PCM  activities.  In  response,  the  SEPG  began 
working  more  closely  with  the  SEI  coach  to  revise  the  PCM  as  needed.  The  SEPG  reassured 
the  RM  IAT  and  other  IATs  that  problems  could  be  worked  out  even  while  using  a  rough, 
prototype  PCM.  The  SEPG  determined  that  it  needed  to  bring  in  the  SEI  coach  early  when 
problems  arose. 

5.7  Lessons  Learned 

The  lessons  listed  here  are  based  on  in-process  as  well  as  post-process  feedback  from  the 
IATs  using  the  PCM,  and  from  the  SEPG 

•  Make  arrangements  with  the  coach  to  be  on  call. 

The  SEPG  and  the  SEI  coach  agreed  that  the  coach  would  be  available  by  phone  to  respond 
immediately  to  questions  that  arose  during  the  RM  and  SPP/SPTO  IAT  meetings. 

•  Assess  IAT  skill  needs  as  early  as  possible. 

It  would  have  been  helpful  to  the  RM  team,  as  they  obtained  root  cause  analysis  training,  to 
have  training  tailored  to  the  SPI  context.  Most  Xerox  personnel  had  been  trained  in  this  and 
other  quality  methods,  but  had  not  used  them  for  SPI-related  problems. 
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6  Activity  5:  Develop  Solution 


6.1  Purpose 

The  purpose  of  Activity  5  was  to  develop  processes,  materials,  and  services  that  would  con¬ 
stitute  a  solution  to  the  problems  identified  in  Activity  4.  Work  in  this  activity  was  based  on 
“whole  product”  principles.  Stated  simply,  this  meant  that  the  process  solution  would  be  de¬ 
livered  to  users  in  projects  with  accompanying  support  services  such  as  communication, 
training  and  consulting,  and  with  documentation,  examples,  templates  and  background  mate¬ 
rial.  Development  of  the  solution  and  collateral  materials  was  to  be  based  on  research  into 
best  (or  good)  practice  elsewhere  and  informal  benchmarking  with  partner  organizations. 
Possible  suppliers  of  commercially  packaged  materials  and  training  were  to  be  tracked  down 
and  screened.  Then  the  solution  would  be  prototyped  and  tried  out,  if  possible,  with  a  project. 
Based  on  the  results  of  this  early  version  and  its  use,  implementation  plans  for  a  final  solution 
would  be  written,  and  the  final  collateral  materials  prepared  in  anticipation  of  a  full-blown 
pilot  evaluation  of  the  solution  in  Activity  6. 

6.2  Tasks 

1 .  Identify  best  practice,  clarify  goals. 

2.  Benchmark  solutions  that  other  organizations  have  used. 

3.  Screen/select  solution  component  sources. 

4.  Design  a  solution. 

5.  Prepare  an  implementation  plan. 

6.  Prepare  implementation  materials. 

7.  Record  and  analyze  lessons  learned. 

6.3  Highlights  of  What  Was  Done 

6.3.1  Special  PCM  Training 

The  RM IAT  and  its  SEPG  consultant  were  given  a  special  half-day  training  session  on  the 
steps  in  Activity  5  by  the  SEI  coach,  because  Activity  5  was  one  of  the  two  most  complex 
activities  in  the  PCM.  In  addition,  because  the  IATs  were  having  difficulty  understanding  the 
PCM  “big  picture,”  the  SEPG  arranged  to  have  a  Xerox  corporate  technical  training  group 
work  with  the  SEI  coach  to  develop  PCM  overview  training.  The  RM  and  the  SPP/SPTO 
IATs  received  this  training. 
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6.3.2  Identifying  Best  Practices  and  Benchmarking 

The  RM  LAT  looked  at  software  tools  that  would  support  requirements  management,  and  also 
purchased  training  from  a  vendor.  The  IAT  developed  a  requirements  management  bench¬ 
marking  questionnaire  and  then  looked  for  benchmarking  partners  both  inside  and  outside 
PSG  West  and  Xerox,  to  compare  notes  on  how  requirements  management  was  done,  and  get 
ideas  for  their  own  solution  design  for  requirements  management.  They  ordered  a  literature 
search  and  hired  a  local  university  professor  to  review  the  search  findings  and  reduce  them  to 
the  best  few  items  for  the  RM  IAT  to  examine.  They  also  performed  their  own  searches  on 
the  World  Wide  Web. 

6.4  What  Worked  Well 

6.4.1  Data  Gathering  Activities 

The  PCM  recommended  looking  both  inside  and  outside  one’s  organization  to  tap  existing 
practice  as  input  to  developing  process  solutions.  Reviewing  conference  proceedings,  doing 
library  and  Internet  searches,  holding  training,  and  tracking  down  experienced  people  were 
all  considered  beneficial  by  the  RM  IAT.  Having  developed  the  benchmarking  questionnaire 
proved  a  helpful  experience,  and  the  questionnaire  expedited  phone  interviews  and  email  in¬ 
quiries  for  this  IAT.  The  SPP/SPTO  IAT,  like  the  RM  IAT,  arranged  for  training  to  be  brought 
in,  investigated  project  management  literature  and  Internet  sites,  attended  seminars,  and  re¬ 
viewed  best  practices  on  existing  projects.  This  IAT  also  drew  heavily  upon  IEEE  standards 
for  project  management.  The  SCM  IAT  was  able  to  draw  from  the  Xerox  SCM  Guide,  and 
this  sped  up  their  progress. 

6.4.2  Building  on  Prior  Know-how 

Because  PSG  West  had  more  experience  with  SCM  than  with  the  other  Level  2  Software 
CMM  KPAs,  there  was  more  consistency  of  understanding  of  this  KPA  among  the  SCM  IAT 
members.  Less  work  was  necessary  to  sort  out  what  was  needed  in  the  process  solution  that 
they  were  developing. 

6.4.3  Need  for  Ongoing  Communication  about  SPI 

As  IATs  moved  into  Activity  5  tasks,  SPI-related  activity  at  PSG  West  intensified,  and  the 
need  for  communication  increased.  SEPG  members  worked  with  PSG  West  managers,  tech¬ 
nical  leads,  and  software  developers  to  get  them  involved  with  and  supportive  of  the  SPI  ef¬ 
fort.  These  individuals  helped  the  improvement  activities  gain  momentum.  Some  were  al¬ 
ready  members  of  IATs  and  provided  information  informally  to  other  members  of  their 
projects.  Some  later  became  subject  matter  experts,  a  formal  role  set  up  at  PSG  West  to  en¬ 
sure  that  SPI-related  information  and  expertise  were  continuously  accessible  to  software  de¬ 
velopers  and  managers. 
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6.4.4  Emergence  of  Solution  Integration  Issues 

Another  outcome  of  LAT  work  converging  in  Activity  5  was  the  recognition  of  the  need  for 
understanding  the  interfaces  between  solutions— that  is,  for  understanding  how  the  solutions 
would  integrate  when  put  into  practice  in  a  project.  LAT  leaders  organized  meetings  to  work 
toward  consistency  among  the  solutions  that  their  teams  were  developing.  The  SQA  team  in 
particular  pushed  for  this,  because  it  was  responsible  for  auditing  projects  using  the  processes 
defined  by  the  KPA  solutions,  and  thus  needed  to  understand  the  interfaces  between  solutions 
and  to  explain  interdependencies  to  those  being  audited. 

6.5  lAT-Level  Issues 

6.5.1  Unevenness  of  PCM  Activities 

The  RM  IAT  suggested  that  a  revised  PCM  have  greater  consistency  among  activities  in  level 
of  effort  and  duration.  For  example,  completion  of  the  first  two  tasks  in  Activity  5  required 
two  calendar  months,  compare  to  only  a  few  weeks  each  for  Activities  1  through  4.  The  SCM 
IAT  said  that  felt  that  it  could  have  skipped  benchmarking;  the  process  did  not  go  well,  in 
part  because  they  were  not  able  to  recruit  many  benchmarking  partners.  In  addition,  they  felt 
they  had  enough  expertise  on  the  team  and  did  not  need  the  external  data. 

The  SCM  IAT  also  said  that  it  wanted  more  training  on  the  Entry  Criteria-Task-Validation- 
Exit  Criteria  (ETVX)  approach  to  process  description,  which  the  PCM  recommended  [Radice 
1988]. 

6.5.2  Scarcity  of  Software  CMM-Specific  Information 

The  RM  IAT  reported  that  it  was  difficult  to  get  information  specific  to  the  RM  KPA  in  Soft¬ 
ware  CMM  Level  2  when  interviewing  benchmarking  partners,  participating  in  training 
courses  from  vendors,  and  while  searching  the  literature.  For  example,  the  search  service  was 
not  familiar  with  the  Software  CMM,  so  it  was  hard  to  get  them  to  focus  their  search  effec¬ 
tively. 


6.6  SEPG-Level  Issues 

All  of  the  RM  IAT’s  problems  with  Activity  5  became  the  SEPG’s  problems,  in  that  revisions 
to  Activity  5  needed  to  be  made  so  that  other  IATs  performing  Activity  5  tasks  would  have 
fewer  issues  to  address.  In  particular,  regarding  benchmarking  as  described  in  Activity  5,  the 
PCM  description  of  the  term  was  unfamiliar  to  the  PSG  West  staff.  The  SEI  coach  was  aware 
of  a  relatively  quick  and  qualitative  (versus  quantitative)  approach  to  benchmarking  and  had 
written  about  this  approach  in  the  PCM  [Spendolini  1992].  However,  given  the  Xerox  cul¬ 
ture,  where  benchmarking  was  an  extensive  and  quantitative  activity,  the  RM  IAT  assumed 
that  the  benchmarking  process  described  in  the  PCM  was  similarly  extensive. 

The  biggest  set  of  issues  for  the  SEPG  to  address  came  in  Activity  5,  as  the  IATs  started  to 
develop  their  process  solutions.  These  issues  included  the  need  for  a  better  understanding  of 
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the  PCM  approach  to  benchmarking,  the  need  for  training  on  the  ETVX  process  notation, 
how  Activity  5  tasks  used  the  outputs  from  Activities  1  through  4,  and  how  to  do  early 
evaluation  of  the  draft  process  solutions.  (Eventually  the  IATs  agreed  to  use  the  ETVX  nota¬ 
tion  as  adapted  by  the  RM IAT.) 

6.7  Lessons  Learned 

•  Identify  and  resolve  any  conflict  in  the  meaning  of  widely  used  terms. 

Terms  such  as  benchmarking  are  widely  known  and  it  is  easy  to  assume  a  common  under¬ 
standing.  The  PCM  version  of  benchmarking  should  have  been  discussed  during  the  pre¬ 
collaboration  revision  of  the  PCM,  and  any  conflict  between  the  PCM  approach  and  the 
Xerox  corporate  approach  should  have  been  noted  and  resolved  at  that  point. 

•  Chunk  the  introduction  of  unfamiliar  strategies ,  processes,  notations ,  etc. 

Activity  5  was  the  first  step  in  the  PCM  that  contained  primarily  unfamiliar  material  and  in 
addition,  it  was  lengthy.  The  IATs  needed  guidance  in  smaller  and  clearer  chunks. 

•  Perform  a  training  needs  analysis  prior  to  initiating  any  improvement  efforts. 

The  PCM  overview  training  was  not  prepared  early  enough  so  that  SEPG  members  had  mate¬ 
rials  for  use  while  training  the  IATs.  The  assumption  was  made  that  the  PCM  document  plus 
the  consulting  of  the  SEPG  member  with  an  IAT,  would  negate  the  need  for  formal  training. 
As  it  was,  the  PCM  document  was  more  helpful  as  a  supplement  to  the  training  that  was 
eventually  developed  than  as  the  main  source  of  guidance  for  the  IATs. 

•  Select  domain  ( subject  matter)  experts  early. 

Most  IATs  saw  the  need  for  domain  expertise  most  clearly  in  Activity  5.  At  that  point,  they 
agreed  that  selecting  a  domain  expert  early,  as  recommended  in  the  PCM,  would  have  been 
helpful.  Each  IAT  could  have  taken  advantage  of  a  domain  expert’s  experience  in  a  number 
of  areas:  for  initial  education  in  a  particular  KPA,  for  assistance  in  baselining  and  determin¬ 
ing  the  desired  process,  for  guidance  in  benchmarking,  and  for  solution  design  and 
prototyping. 
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7  Activity  6:  Pilot  Use  and  Evaluation 


7.1  Purpose 

Activity  6  focused  on  piloting  the  process  solution  and  the  related  materials  and  services  that 
were  developed  in  Activity  5  by  an  IAT  for  a  particular  KPA.  The  purpose  of  piloting  was  to 
validate  a  solution  prior  to  deployment  in  the  project  community.  Piloting  was  the  responsi¬ 
bility  of  the  IATs,  but  the  SEPG  provided  assistance  with  this  task.  Note  that  while  the  IATs 
were  staffed  by  people  who  were  spending  at  least  80%  of  their  time  on  project  work,  the 
piloting  process  was  their  first  opportunity,  as  IAT  members,  to  engage  the  PSG  West  projects 
and  determine  if  the  work  they  had  done  was  on  target. 

7.2  Tasks 

1 .  Prepare  the  pilot  plan  and  materials. 

2.  Initiate  the  introduction  sequence  and  begin  pilot  use. 

3.  Monitor  the  pilot  projects  and  record  issues  and  problems. 

4.  Evaluate  results  of  the  piloting  process  and  determine  next  steps. 

5.  Record  and  analyze  lessons  learned. 

7.3  Highlights  of  What  Was  Done 

7.3.1  Planning  for  Pilots 

The  SEPG  created  a  plan  for  the  collective  IAT  piloting  process.  This  plan  contained  a  list  of 
prospective  pilot  projects  with  their  schedule  status;  for  each  pilot,  notes  indicated  where  it 
might  be  appropriate  to  intercept  the  project  in  order  to  pilot  solutions.  The  SEPG  also  cre¬ 
ated  a  template  pilot  plan  which  the  IATs  used  to  plan  each  of  their  pilots.  The  template  pro¬ 
vided  a  consistent  starting  point  for  each  IAT. 

There  were  three  important  problems  to  solve  in  planning  the  pilots,  which  the  SEPG  led  the 
process  of  addressing: 

1 .  How  do  we  pilot  KPA-focused  solutions  in  projects? 

2.  How  do  we  handle  piloting  multiple  solutions  in  projects? 

3.  How  do  we  handle  management  sponsorship  of  piloting  solutions  in  projects? 
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7.3.2  The  Initial  Piloting  Strategy 

The  initial  approach  to  the  piloting  strategy  was  based  on  implementing  an  entire  IAT  solu¬ 
tion  within  a  single  project.  The  RM  IAT,  for  example,  had  planned  to  work  closely  with  any 
project  that  agreed  to  act  as  a  pilot.  This  strategy  required  the  IAT  to  complete  its  solution 
before  the  pilot  could  begin.  (The  adoption  of  the  idea  of  prototyping  partial  solutions  in  col¬ 
laboration  with  projects  during  Activity  5  came  too  late  for  most  IATs  to  benefit;  see  discus¬ 
sion  below  in  9.4).  The  assumed  benefit  of  this  approach  was  that  at  the  end  of  the  pilot  the 
project  would  have  already  implemented  the  IAT  solution,  and  roll  out  would  then  not  be 
necessary  for  the  project  involved.  Also,  the  IAT  would  be  able  to  test  its  entire  solution.  As  it 
turned  out,  most  IATs  did  not  pilot  in  this  manner. 

7.3.3  Criteria  for  Selecting  Pilot  Projects 

The  material  in  the  pilot  plan  template  provided  a  basis  for  selecting  projects  to  participate  in  pi¬ 
loting.  Each  IAT  created  a  pilot  plan  based  on  this  template  and  adopted  the  following  criteria: 

•  Project  is  in  the  appropriate  development  life-cycle  phase  to  support  the  KPA. 

•  Project  schedule  can  accommodate  the  introduction  and  completion  of  a  pilot. 

•  Project  pilot  team  contains  the  appropriate  skill  levels  to  execute  and  evaluate  the 
solution. 

•  Project  is  representative  of  PSG  West  software  development  projects. 

•  Project  should  not  be  in  a  recovery  mode  (that  is,  not  in  the  process  of  recovering  from  a 
difficult  situation). 

•  Project  must  be  able  to  implement  the  solution  without  much  difficulty  or  risk,  and 
implement  it  within  the  pilot  schedule. 

•  Project  is  of  an  appropriate  size  to  execute  and  evaluate  the  solution. 

•  Project  is  appropriate  for  piloting  the  specific  solution. 

•  Selection  of  this  project  contributes  to  a  good  mix  of  pilot  projects  from  across  the 
division. 

•  Project  personnel  want  to  do  the  pilot  and  there  are  resources  to  staff  a  pilot  project  team 
to  work  with  the  SEPG  and  the  IAT. 

Using  these  criteria  as  guidelines,  the  SEPG  and  the  MST  reviewed  projects  at  PSG  West  and 
determined  which  were  candidates.  Projects  were  then  contacted  by  the  SEPG  to  discuss  their 
interest  in  participating  in  a  pilot  and  the  resources  they  had  available  for  participating. 

7.3.4  Pilot  Sponsorship 

The  pilot  strategy  included  the  concept  that  MST  members  would  be  sponsors  of  the  pilots: 
many  but  not  all  of  the  managers  of  PSG  West  projects  were  members  of  the  MST.  By  spon¬ 
soring  the  pilots,  MST  members  were  endorsing  the  efforts  of  the  IATs  and  the  SEPG.  In 
practice,  the  current  business  processes  were  given  higher  priority  than  piloting  the  KPA  so- 
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lutions,  so  the  strategy  was  not  as  successful  as  expected.  Also,  the  selection  criteria  for  pi 
lots  provided  a  strong  filter  that  limited  the  projects  available  for  pilots. 

7.3.5  Dealing  with  Project  Schedules  and  Deadlines:  Piloting 
a  Composite  of  the  Solution 

The  initial  strategy  of  implementing  the  complete  KPA  process  solution  during  piloting  proved 
difficult  because  projects  were  asked  to  absorb  the  overhead  of  piloting  without  any  schedule  or 
resource  relief.  Given  that  time  and  some  level  of  resources  were  essential  to  implement  a  solu¬ 
tion,  the  IATs  decided  to  alter  their  strategy  to  pilot  partial  solutions  with  the  projects. 

This  approach  meant  that  multiple  pilots  were  needed  to  test  the  solution  completely.  This 
strategy  had  a  smaller  impact  on  each  project,  and  the  project  managers  were  more  willing  to 
accept  the  pilots.  In  retrospect  this  is  not  surprising,  because  all  of  these  new  processes  were 
developed  externally  to  the  projects  and  were  perceived  as  a  major  risk  by  project  personnel. 
The  IATs,  on  the  other  hand,  had  to  use  more  time  and  effort  in  engaging  and  supporting 
multiple  pilots,  and  this  delayed  the  final  release  of  the  solutions. 

7.3.6  Piloting  Multiple-KPA  Solutions 

Some  IATs  determined  that  they  needed  to  work  together  to  pilot  their  solutions.  For  exam¬ 
ple,  the  SQA IAT  identified  the  need  to  work  with  the  SPP/SPTO IAT  to  validate  its  own  so¬ 
lution,  which  described  the  process  for  helping  prepare  and  review  a  project’s  software  de¬ 
velopment  plan.  The  SQA  IAT  needed  to  test  its  process  in  an  environment  where  the 
software  development  plan  template  created  by  the  SPP/SPTO  IAT  was  being  used,  and  these 
IATs  began  working  together  to  engage  a  project  jointly  to  achieve  this  goal. 

This  evolution  of  the  piloting  strategy  proved  extremely  difficult  to  achieve  in  practice  be¬ 
cause  once  again  the  overhead  on  the  project  and  project  deadlines  became  an  issue.  Just  one 
project  agreed  to  pilot  multiple  KPAs  and  negotiated  with  the  IATs  to  pilot  only  selected 
pieces  of  each  KPA  solution  to  minimize  the  resources  needed. 


7.4  What  Worked  Well 

7.4.1  Project  Participation  in  Piloting 

Most  projects  participating  in  pilots  appointed  a  team  of  their  own  staff  (the  PCM  called  them 
Local  Change  Teams  [LCTs])  as  a  type  of  mini-,  project-specific  IAT — to  work  with  PSG 
West  IATs  to  customize  KPA  solutions  and  apply  them.  Communication  among  the  MST, 
IATs,  and  piloting  projects  occurred  frequently.  During  monthly  status  meetings  with  the 
MST,  the  SEPG  discussed  plans,  status  against  plans,  issues  and  opportunities,  and  budgets. 
Each  SEPG  consultant  assigned  to  an  IAT  and  piloting  project  communicated  informally  with 
them  on  a  weekly  basis,  addressing  plans,  progress  and  roadblocks. 
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7.4.2  Revising  the  Solutions 

The  IATs  were  extremely  thorough  in  developing  their  KPA  solutions,  and  this  became  ap¬ 
parent  during  the  pilots.  None  of  the  pilots  provided  input  for  significant  changes  to  the  KPA 
solutions.  Although  the  IATs  did  not  pilot  their  entire  solutions,  the  fact  that  so  few  changes 
resulted  from  the  pieces  that  were  piloted  provided  a  degree  of  confidence  in  the  remaining 
pieces  of  the  solutions.  In  retrospect,  one  way  to  look  at  the  solutions  that  were  developed  is 
to  view  them  as  the  result  of  an  extensive  organizational  learning  process.  Even  while  some 
projects  did  not  adopt  the  solutions  intact,  or  adapted  them  heavily,  the  expertise  accumulated 
by  IAT  members,  who  were  drawn  from  throughout  PSG  West,  was  broadly  available  to 
serve  as  the  basis  for  a  variety  of  approaches  to  solutions. 

7.4.3  Solution  Hand  Off  After  Piloting 

Once  the  IATs  completed  piloting  their  solutions  and  making  adjustments  based  on  the  results 
of  the  pilots,  they  were  ready  to  provide  the  solutions  to  the  SEPG  for  inclusion  in  the  proc¬ 
ess  asset  library.  The  IATs  and  the  SEPG  all  agreed  that  it  was  appropriate  for  the  IATs  to 
present  the  final  work  to  the  MST  for  approval  before  giving  it  to  the  SEPG  for  publishing 
online.  Once  again  the  RM  IAT  took  the  lead,  and  prepared  a  brief  slide  presentation  as  well 
as  a  package  for  handing  out  to  the  MST  members.  The  package  contained  the  requirements 
management  process  description,  templates  and  job  aids.  It  also  included  a  table  with  cross 
references  of  the  Software  CMM  requirements  management  practices  with  the  RM  process 
description  showing  where  the  process  description  addressed  each  of  the  practices.  In  accor¬ 
dance  with  the  PCM,  the  package  also  contained  the  lessons  learned  for  the  whole  RM  IAT 
activity.  The  other  IATs  later  adopted  this  format. 

7.5  lAT-Level  Issues 

Complete  process  solutions  could  not  be  piloted  on  a  single  project.  Coordination  across  IATs 
in  Activity  6  thus  became  even  more  necessary  than  it  was  perceived  to  be  in  Activity  5. 

IATs,  especially  the  RM  IAT  in  leading  the  way,  became  aware  of  how  important  piloting 
really  was  for  making  change  work.  One  RM  IAT  member  noted,  in  reflecting  on  the  experi¬ 
ence  of  piloting  the  new  requirements  management  process  solution,  that  there  would  be  a 
painful  period  of  time  while  old  habits  were  being  replaced  with  new  ones.  The  RM  IAT  felt 
it  was  helpful  to  explain  to  the  piloting  project  that  this  painful  transition  was  to  be  expected, 
and  that  the  IAT  members  would  support  project  members  as  they  went  through  it.  Another 
RM  IAT  member  noted,  “The  RM  process  is  fine.  Implementation  of  the  process  is  the  ques¬ 
tion  and  depends  upon  the  organization.”  The  PCM  attempted  to  describe  the  difference  be¬ 
tween  the  process  that  was  inherent  in  the  solution,  and  the  process  for  the  introducing  or 
implementing  of  the  solution  in  a  project,  and  the  need  for  repeatability  of  both.  During  pi¬ 
loting,  the  RM  IAT  discovered  first-hand  why  this  was  the  case. 
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7.6  SEPG-Level  Issues 

During  the  period  when  IATs  were  most  involved  in  piloting,  the  membership  of  the  SEPG 
was  evolving.  Newer  members  were  not  yet  completely  familiar  with  the  PCM  and  how  the 
IATs  were  using  it,  and  this  may  have  limited  the  effectiveness  of  PCM  use  for  piloting. 

7.7  Lessons  Learned 

•  Get  sponsorship,  as  well  as  funding  relief,  for  projects  piloting  the  IAT  solutions. 

Schedule  relief  is  not  possible  in  a  market-driven  organization.  Thus  getting  senior  manage¬ 
ment  to  provide  projects  with  resource  relief  by  underwriting  the  risk  out  of  special  funds 
may  give  IATs  a  firmer  basis  for  negotiating  pilot  participation.  This  is  also  difficult,  but  es¬ 
sential,  and  needs  to  be  arranged  during  SPI  strategy  and  plan  development  as  well  as  again 
when  piloting  becomes  imminent.  Providing  an  estimate  of  the  nature  and  amount  of  re¬ 
sources  required  from  the  project  may  help  in  the  negotiations  with  both  senior  managers  and 
project  managers. 

•  Plan  on  piloting  a  series  of  partial  solutions  for  a  given  KPA. 

Piloting  a  complete  solution  all  at  once  is  very  likely  impossible,  so  planning  to  pilot  partial 
solutions  is  a  good  idea.  Also,  anticipate  the  need  to  pilot  in  conjunction  with  at  least  one 
piece  of  a  solution  for  another  KPA;  this  mirrors  the  complex  situations  on  projects  in  the 
“real”  world. 

•  Anticipate  difficulty  during  pilot. 

Helping  project  organizations  to  make  process  changes,  while  projects  are  underway  devel¬ 
oping  and  delivering  product,  is  very  difficult.  Plan  for  pilots  the  same  way  you  would  plan 
for  working  with  an  external  customer  to  try  out  a  new  product.  During  the  pilot  pay  careful 
attention  to  the  steps  used  in  the  introduction  of  the  solution;  these  should  be  refined  as  you 
work  with  a  range  of  projects  until  they  are  highly  repeatable.  After  each  significant  piloting 
event,  and  especially  during  the  first  IATs  piloting  process,  the  SEPG  and  representatives  of 
all  the  IATs  should  debrief  that  IAT,  to  reduce  risk  for  later  IAT  piloting  work. 

•  Draw  IAT  members  from  the  broadest  range  of  projects  that  are  candidates  to  act  as 
pilots. 

Drawing  IAT  members  from  a  broad  range  of  projects  pays  off  during  piloting.  At  PSG  West, 
the  broad  organizational  learning  represented  by  the  collectively  gained  experience  of  IAT 
members  became  the  basis  not  only  for  successful  piloting  but  also  for  later  roll  out  (broad 
deployment)  of  the  solutions.  Some  IAT  members  supported  early  use  of  parts  of  solutions  in 
their  home  projects  prior  to  any  “official”  use  during  piloting  or  roll  out. 
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8  Activity  7:  Roll  Out 


8.1  Purpose 

The  purpose  of  Activity  7  was  to  facilitate  the  adaptation  and  adoption,  by  each  PSG  West 
project,  of  the  solutions  that  were  developed  in  Activity  5  and  piloted  in  Activity  6.  As  origi¬ 
nally  envisioned  in  the  PCM,  Activity  7  was  to  be  performed  by  the  LATs  with  ongoing  sup¬ 
port  from  the  SEPG  and  from  other  organizational  functions  such  as  training.  In  addition,  the 
PCM  proposed  using  an  LCT  on  each  project,  to  work  with  the  IATs  in  performing  the  adap¬ 
tation  for  the  solutions.  As  implemented  in  PSG  West,  Activity  7  was  solely  the  responsibility 
of  the  SEPG  to  whom  the  LATs  had  handed  off  their  process  solutions  and  collateral  materials 
at  the  end  of  Activity  6  work.  And  as  in  Activity  6,  the  LCTs  were  sometimes  implemented 
formally,  sometimes  informally,  and  with  various  names. 

8.2  Tasks 

1 .  Design  and  negotiate  ongoing  support,  including  monitoring  and  improving  mechanisms 
that  support  the  solutions. 

2.  Prepare  a  roll-out  plan. 

3.  Prepare  for  introduction  to  project  n. 

4.  Initiate  roll-out  plan  in  project  n. 

5.  Monitor  and  improve  the  solutions. 

6.  Record  and  analyze  lessons  learned  and  wrap  up. 

8.3  What  Worked  Well 

8.3.1  How  the  MST  Participated  in  Roll  Out  Planning 

The  MST  participated  in  an  all-day  off-site  meeting  to  give  input  on  which  projects  should 
participate  in  the  roll-out  process  and  to  assess  the  goals  and  objectives  of  roll  out  as  articu¬ 
lated  by  the  SEPG  in  the  Activity  7  description. 

8.3.2  Staggered  Approach  to  Roll  Out  of  Solutions 

The  SEPG  created  a  plan  that  outlined  rolling  out  of  piloted  and  revised  solutions  to  software 
projects  in  a  phased  manner.  This  approach  was  selected  because  the  SEPG  could  effectively 
support  only  one  project  at  a  time.  The  MST  approved  the  roll-out  sequence  based  on  the 
need  and  current  status  of  these  projects. 
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8.3.3  Communication  about  Roll  Out  of  Solutions 

Dissemination  of  the  solutions  began  with  a  communication  memo  sent  to  all  members  of 
PSG  West  software  projects,  notifying  them  of  the  availability  of  the  process  solutions.  While 
the  KPA  process  solutions  were  being  developed  by  the  IATs,  the  SEPG  established  a  direc¬ 
tory  structure  on  a  file  server  in  the  Xerox  internal  network  for  storing  of  the  processes  and 
any  IAT-related  materials  such  as  minutes  of  meetings.  This  area  was  in  effect  the  early  proc¬ 
ess  asset  library,  and  eventually  evolved  into  a  website.  Its  main  purpose  at  the  time  was  to 
provide  a  common  area  for  use  by  the  IAT  and  SEPG  while  processes  were  being  developed. 

8.4  lAT-Level  Issues 

The  transition  from  piloting  in  Activity  6  to  roll  out  in  Activity  7  was  not  clear  cut.  It  was 
difficult  to  determine  criteria  for  completing  pilots,  since  piloting  of  solutions  occurred  across 
projects,  with  only  partial  deployment  of  solutions  in  each.  As  discussed  earlier,  there  were 
also  issues  of  integration  among  solutions  that  needed  to  be  worked  out;  the  SQA  IATs  own 
solution  depended  on  resolving  these. 

8.5  SEPG-Level  Issues 

The  SEPG  understood  conceptually  and  strategically  what  had  to  happen  in  Activity  7.  But 
lacking  practical  experience,  and  at  the  same  time  working  to  support  the  IATs  in  the  prob¬ 
lems  just  described,  the  SEPG  had  its  own  learning  curve  to  climb.  The  SEPG  decided  to  cre¬ 
ate  a  roll-out  plan  for  all  of  the  KPA  solutions  created  by  the  IATs,  which  included  a  schedule 
of  which  solutions  to  introduce  to  which  projects  and  when.  Included  in  the  plan  was  a  set  of 
prerequisites  to  roll-out  tasks:  having  an  SEPG  hotline  established,  having  the  process  asset 
library  ready,  and  having  a  subject  matter  expert  network  set  up.  Implicit  in  all  of  this  was  the 
decision  that  the  SEPG  would  indeed  be  responsible  for  Activity  7 ;  the  IATs  were  to  be  re¬ 
tired  as  they  handed  off  their  completed  solutions. 

8.6  Lessons  Learned 

•  LCTs  or  their  equivalent  are  essential  to  successful  roll  out  of  solutions. 

As  projects  came  nearer  to  applying  the  process  solutions  in  practice,  the  SEPG  saw  the  need 
to  establish  project-based  teams  to  work  on  project-specific  versions  of  IAT  solutions — and 
began  to  establish  these,  if  not  always  labeling  the  teams  as  LCTs.  These  teams  were  consti¬ 
tuted  somewhat  differently  from  project  to  project,  but  generally  included  a  manager  as  either 
a  full  time  or  part-time  team  leader,  and  part-time  project  members  as  representatives  of  the 
areas  affected  by  the  solutions.  The  teams  worked  to  translate  the  general  solutions  from  the 
IATs  to  the  specific  situation  in  the  projects.  In  some  cases,  they  created  their  own  solutions 
and  did  not  use  those  of  the  IATs. 

This  meant  that  the  biggest  change  in  SPI  infrastructure  during  roll  out  was  within  the  proj¬ 
ects  themselves.  The  LCT  was  responsible  for  planning  and  implementing  SPI  within  the 
project,  and  took  on  the  role  of  communicating  local  SPI  information  to  project  members. 
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The  SPI  activities  for  the  project  were  defined  in  the  SPI  implementation  plan  that  identified 
and  defined  the  roles  of  the  project-based  team  and  defined  a  schedule  for  each  of  the  SPI 
activities.  This  plan  was  the  road  map  for  SPI  implementation  within  the  project. 

•  Plan  to  evolve  the  SEPG  role  during  roll  out. 

During  roll  out,  the  role  of  the  SEPG  members  also  changed,  from  advisors  and  facilitators  of 
the  IATs,  to  advisors  and  facilitators  of  the  projects  as  they  implemented  SPI.  The  SEPG  be¬ 
came  the  keeper  of  the  process  solutions  and  maintained  these  processes  via  a  documented 
change  process.  The  MST  members  continued  to  meet,  although  less  frequently,  and  also 
continued  with  their  role  as  SPI  champions.  Within  the  projects  that  they  managed,  they  en¬ 
sured  that  SPI  implementation  continued. 
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9  Activity  8:  Wrap  Up 


9.1  Purpose 

The  purpose  of  Activity  8  was  to  evaluate  the  work  of  the  first  seven  activities,  determining 
where  the  PCM  could  be  adapted  and  improved  for  its  use  in  another  change  effort.  The  PSG 
West  evaluation,  facilitated  by  the  SEPG  included  interviewing  IAT  members  for  their  final 
feedback  and  lessons  learned,  coming  to  closure  on  all  tasks,  publishing  results,  and  recog¬ 
nizing  the  efforts  of  IAT  members  and  others. 

9.2  Tasks 

1 .  Analyze  the  lessons  learned  and  wrap  up. 

2.  Conduct  the  final  lessons  learned  analysis. 

3.  Conduct  a  checkpoint  review. 

4.  Publish  the  results. 

5.  Recognize  the  efforts  of  IAT  members. 

9.3  Highlights  of  What  Was  Done 

The  SEPG  debriefed  all  of  the  IATs,  obtaining  lessons  learned  from  each.  The  SEPG  also  re¬ 
viewed  the  lessons  learned  reports  that  had  been  provided  by  each  IAT  at  the  end  of  each 
PCM  step.  The  SEI  coach  interviewed  representatives  of  the  SPI  effort,  including  MST 
members,  SEPG  members,  IAT  members  and  LCT  or  other  project  team  members.  The 
authors  of  this  report  represent  the  SEPG  and  the  SEI,  and  wrote  this  report  as  documentation 
of  the  analysis  of  the  PCM  collaboration. 

9.4  Lessons  Learned  in  Activities  1-7 

These  lessons  reflect  on  the  entire  set  of  PCM  activities,  including  the  areas  of  the  “lead 
goose”  strategy,  solution  integration,  piloting,  training,  IAT  size,  and  IAT  duration. 

•  Plan  on  multiple  partial  pilots  of  solutions  and  work  with  piloting  projects 
opportunistically. 

The  process  of  piloting  IAT  solutions  led  to  an  improved  understanding  of  piloting.  The  PCM 
did  not  address  multiple  and  partial  pilots.  The  original  version  of  the  PCM  recommended 
one  complete  pilot. 
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As  the  IATs  began  to  develop  solutions  and  to  try  to  pilot  them,  they  discovered  that  projects 
participating  in  pilots  wanted  the  relationships  among  the  solutions  documented  and  evalu¬ 
ated.  These  projects,  busy  trying  to  complete  product  development,  did  not  have  time  to  de¬ 
termine  the  interfaces  between  the  solutions  themselves.  (At  PSG  West,  the  SQA IAT  led  the 
effort  to  work  on  these  solution  integration  issues  because  its  solution  was  affected  by  all  of 
the  other  IAT  solutions:  this  IAT  had  to  check  and  facilitate  compliance.)  What  appeared  to 
work  best  was  for  piloting  projects  to  adopt  a  piece  of  a  solution,  and  then  “digest”  that  prior 
to  taking  on  the  next  piece.  Integration  issues  were  solved  in  a  just-in-time-manner  when 
projects  were  ready  for  the  next  solution  piece.  Most  IATs  partially  piloted  their  solutions  on 
two  projects. 

This  approach  might  seem  inconsistent  with  the  idea  of  developing  complete,  KPA-specific 
solutions.  However,  the  solution  development  process  created  a  knowledge  base  that  both  the 
IAT  and  the  SEPG  drew  upon,  during  piloting  and  also  during  roll  out. 

•  Provide  training  and  job  aids. 

It  was  expected  by  both  the  SEI  and  PSG  West  that,  with  the  PCM  being  a  prototype,  a  great 
deal  of  direct  support  to  IATs  would  be  needed,  and  this  was  routinely  provided  via  the  SEPG 
consultant  assigned  to  each  IAT.  However,  at  several  points  during  the  collaboration,  it  was 
clear  that  more  formal  training  would  also  have  been  helpful.  IAT  members  expressed  con¬ 
cern  that  terminology  used  in  the  PCM  was  not  consistent  with  terminology  used  in  SPI  and 
Software  CMM  orientation  and  training.  IATs  also  indicated  that  they  thought  the  PCM  “big 
picture”  was  missing.  A  better  overview  of  the  overall  flow  of  the  PCM  and  additional  detail 
at  the  beginning  of  each  activity  would  have  helped  IATs  understand  the  transition  from  the 
previous  activity.  Brief,  formal  training  of  IATs  as  they  began  use  of  the  PCM,  and  as  they 
proceeded,  would  also  have  been  helpful. 

The  final  area  where  additional  PCM  training  for  the  IATs  was  identified  was  the  “EVTX” 
notation  used  to  describe  PCM  processes.  This  notation  was  not  described  in  enough  detail  in 
the  PCM  for  IATs  to  use  themselves  without  training. 

In  areas  not  related  to  the  PCM  itself,  some  IATs  drew  upon  Xerox  internal  training  resources 
for  quality-related  courses,  and  obtained  training  in  team  building,  benchmarking,  and  root 
cause  analysis.  Most  IATs  obtained  KPA-related  training  from  vendors,  to  build  on  the  brief 
orientation  given  them  by  the  SEPG  at  kick  off. 

In  addition  to  training  for  IATs,  training  for  members  of  the  MST  and  for  sponsors  of  pilot 
projects  and  even  for  senior  management  is  recommended.  The  PCM  addressed  keeping 
sponsors  informed  but  did  not  specifically  call  for  training  or  orientation  for  sponsors  or  the 
MST.  In  most  cases,  managers  involved  in  SPI  had  limited  exposure  to  SPI  concepts  and  ex¬ 
perience  because  of  time  constraints.  Brief  training  (perhaps  2  hours)  of  these  managers  at 
key  points  would  be  helpful  to  both  them  and  those  they  manage. 
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In  general,  training  all  those  involved  to  an  appropriate  level  of  knowledge  and  skill  should 
be  part  of  rolling  out  solutions;  IATs  should  consider  working  early  in  solution  development 
with  internal  training  personnel  to  get  assistance  with  training  design  and  development,  and  if 
possible  hand  off  the  ongoing  training  job  to  internal  training  organizations. 

•  Create  IATs  that  are  representative  and  adequately  sized. 

The  PCM  provided  detailed  guidance  on  selecting  IAT  leaders,  but  did  not  offer  guidance  on 
selection  of  members  or  numbers  of  members.  PSG  West’s  experience  indicates  that  IATs 
should  have  members  representing  most  (if  not  all)  projects  that  will  be  involved  in  a  major 
change  effort  like  SPI.  This  broad  representation  “seeds”  those  projects  with  early  expertise 
in  the  IATs  area  of  focus.  In  addition,  having  adequate  size  allows  for  team  attrition  while 
assuring  maintenance  of  a  critical  mass  of  members.  Some  IATs  at  PSG  West  with  greater 
numbers  of  members  (8  to  12)  seemed  to  be  better  able  to  maintain  momentum,  as  loss  of 
members  or  intermittent  attendance  had  less  impact.  Other  teams  seemed  to  evolve  so  that  a 
core  group  emerged,  in  effect  acting  as  co-leaders;  this  was  also  a  successful  strategy  but 
more  vulnerable  to  member  availability. 

•  Set  up  IATs  that  require  full  time  membership. 

The  PCM  did  not  specify  how  IATs  should  do  work,  in  terms  of  either  amount  or  frequency. 
But  one  result  of  the  SEI/PSG  West  collaboration  was  to  make  clear  that  providing  several 
working  models  for  teams,  with  pros  and  cons  for  each,  would  have  been  helpful.  Without 
these  as  possible  alternatives,  PSG  West  chose  to  organize  its  teams  in  a  well-respected, 
quality  methodology-based  approach. 

But  SPI  was  new  to  PSG  West  (indeed,  new  in  general)  and  feedback  from  the  IATs  indicated 
that  10  hours  per  week  was  not  enough  time  for  them  to  accomplish  their  charters.  They  also 
indicated  that  part-time  IAT  work  made  it  difficult  to  sustain  momentum  and  make  progress. 
At  PSG  West,  meeting  less  than  weekly  was  problematic  even  as  early  as  Activity  1:  not  eve¬ 
ryone  could  attend  each  meeting,  and  efficiency  was  lost  as  the  IAT  needed  time  to  re¬ 
establish  context  and  regain  momentum  at  each  meeting.  IAT  members  reported  performing 
many  tasks  outside  of  normal  working  hours  and  in  addition  to  their  usual  project  tasks.  One 
alternative  to  part-time  membership  suggested  by  a  PSG  West  IAT  member  was  to  convene 
an  IAT  for  a  period  of  time  of  perhaps  one  month,  moving  project  members  to  full-time  IAT 
work  temporarily.  By  not  having  to  split  their  attention,  members  could  focus  on  IAT  work 
and  expedite  solution  development  and  piloting.  (This  approach  may  not  always  be  feasible.) 
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10  The  Value  of  the  Collaboration 


The  SEI  and  PSG  West  both  gained  value  from  the  collaboration;  both  also  took  risks  and  had 
disappointments.  Collaboration  per  se  is  inherently  risky— it  implies  willingness  to  work  to¬ 
gether  where  neither  party  has  the  last  word.  It  also  implies  that  those  engaged  in  the  collabo¬ 
ration  are  approaching  it  from  different  perspectives  and  capabilities,  and  thus  that  communi¬ 
cation  across  those  perspectives  may  at  times  be  difficult.  This  was  certainly  true  for  the 
collaboration  described  here.  In  addition,  there  was  the  added  complexity  of  the  subject  of  the 
collaboration  being  an  immature  product  (the  PCM),  and  the  context  of  the  collaboration  be¬ 
ing  a  major  SPI  effort. 

The  PCM  described  how  IATs  could  plan  and  manage  change  efforts  in  PSG  West  software 
organizations  with  the  assistance  of  the  SEPG.  It  provided  a  step-by-step  process  for  intro¬ 
ducing  software  technology,  including  processes,  methods,  and  tools  that  support  imple¬ 
menting  KPAs  of  the  Software  CMM.  When  the  SEPG  began  planning  SPI  following  its  as¬ 
sessment,  it  looked  for  guidance  on  how  best  to  approach  SPI;  they  anticipated  that  the  PCM 
offered  the  benefit  of  IATs  not  having  to  develop  their  strategies  and  plans  from  scratch,  and 
of  savings  in  time  and  effort. 

The  reaction  to  the  PCM  was  mixed.  Some  SEPG  members  liked  the  structure  provided  by 
the  PCM.  The  IAT  members  disliked  the  lack  of  maturity  of  the  PCM.  The  PCM  was  useful 
in  providing  an  overall  road  map  for  the  IAT  members,  but  it  was  not  as  helpful  to  IATs  with 
the  details  of  developing,  piloting,  and  deploying  their  solutions.  And  the  SEPG  members, 
serving  as  consultants  to  the  IATs  using  the  PCM,  were  not  yet  expert,  as  they  were  also  new 
to  the  PCM. 

One  area  where  the  IATs  felt  that  the  PCM  fell  short  was  in  KPA-specific  examples  of  proc¬ 
esses  and  artifacts  such  as  software  project  plans  or  SQA  plans.  It  is  now  common  for  organi¬ 
zations  within  the  software  community  to  purchase  processes,  often  with  example  artifacts, 
and  modify  them.  There  were  none  available  at  the  time  PSG  West  was  working  on  the  SPI 
effort  described  here,  with  the  exception  of  the  SPF,  which  had  no  accompanying  artifacts. 

If  a  “lead  goose”  strategy  is  used  to  initiate  a  group  of  IATs,  then  the  lead  IAT  needs  adequate 
time.  At  PSG  West,  the  RM  IAT  had  the  most  to  learn,  being  first.  Each  team  that  followed 
RM  worked  more  quickly,  and  eventually  the  IATs’  schedules  were  nearly  concurrent.  Start¬ 
ing  the  RM  IAT  an  additional  four  to  six  weeks  early  would  have  allowed  more  time  for  the 
SEPG  to  learn  from  and  respond  to  that  IATs  experience.  Revisions  not  only  to  the  PCM  but 
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also  to  templates  and  other  materials  could  have  been  made  available  to  the  other  IATs  in  a 
more  timely  way. 

The  result  of  the  PCM  having  limited  artifacts  was  that  the  SEPG  and  IATs  themselves  cre¬ 
ated  what  was  needed.  The  PSG  West  SEPG  had  resources  for  creating  draft  templates,  for 
example  the  Pilot  Plan  template  used  by  the  IATs,  and  was  also  able  to  draw  upon  a  Xerox 
technical  communication  group  to  refine  the  template  and  make  it  easier  to  use.  The  template 
was  then  used  to  bootstrap  the  creation  of  examples:  it  required  extra  effort  on  the  part  of  the 
first  IAT  completing  the  template,  but  the  resulting  plan  became  an  example  that  all  the  other 
IATs  could  follow.  While  these  templates  were  proprietary  and  not  available  for  public  use, 
the  strategy  of  creating  a  template  and  bootstrapping  an  example  was  a  valuable  output  from 
the  collaboration  which  could  be  widely  shared  by  the  SEI. 

A  theme  throughout  the  use  of  the  PCM  was  that  the  IATs,  understandably,  wanted  “how”  as 
well  as  “what.”  That  is,  they  wanted  PSG  West-specific  guidance  as  well  as  KPA-specific 
guidance.  The  essence  of  the  PSG  West/SEI  collaboration  was  that  the  SEI  would  provide  the 
“what”  and  would  work  with  PSG  West  to  develop  a  PSG  West-specific  “how.”  PSG  West 
people  knew  their  own  context,  and  SEI  could  help  them  articulate  this  and  use  it  as  they  ap¬ 
plied  the  PCM.  But  the  IATs  needed  specific  “hows”  faster  than  this  collaborative  process 
could  accommodate.  This  experience  provides  a  good  argument  for  why  it  might  have  been 
preferable  to  apply  the  PCM  to  one  IAT,  and  to  have  that  IAT  precede  the  others  by  several 
months.  Doing  this  would  have  allowed  for  revising  the  PCM  to  include  more  PSG  West- 
specific  material  and  approaches  prior  to  the  main  SPI  effort.  However,  this  approach  may 
not  viable  due  to  the  priority  of  getting  products  to  market. 

Altogether,  most  SEPG  and  IAT  members  agreed  that  the  PCM,  offering  a  common  approach 
to  a  complex,  unfamiliar  process  of  introducing  change,  was  useful  despite  the  limitations  in 
structure,  content,  granularity,  usability,  and  format  identified  by  the  RM  and  others.  In  addi¬ 
tion,  many  discussions  were  held  over  the  course  of  the  collaboration  regarding  how  the 
PCM  might  have  been  more  effectively  used;  the  most  common  problem  seemed  to  be  that 
the  IATs,  with  no  experience  as  change  agents,  attempted  to  follow  the  PCM  too  carefully.  A 
major  benefit  of  the  PCM,  given  the  concurrent  activity  of  the  IATs,  was  a  common  vocabu¬ 
lary  for  the  work  they  were  doing.  This  was  used  in  communicating  among  teams  and  also 
provided  a  common  basis  to  status  reporting. 

The  SEI  coach  and  others  from  the  SEI  found  the  experience  of  evaluating  the  prototype 
PCM  in  the  context  of  an  actual  SPI  effort  invaluable.  The  feedback  just  described  was  re¬ 
ceived  during  periodic  on-site  visits  and  regular  teleconferences  as  well  as  at  the  end  of  the 
collaboration  and  was  specific  to  the  PCM.  This  was  extremely  useful.  In  addition,  the  op¬ 
portunity  to  work  closely  with  an  organization  performing  SPI  allowed  the  SEI  to  observe 
activities  that  were  not  addressed  in  the  PCM.  For  example,  the  PSG  West  SEPG  did  an  ex¬ 
cellent  job  of  managing  SPI-related  communications  across  groups  and  levels  within  the  or¬ 
ganization.  The  PCM  addressed  communication  only  with  regard  to  specific  KPA-related  or- 
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ganization  changes.  Inventive  approaches  such  as  the  round  table  meetings  for  gathering  pro¬ 
cess  requirements  should  be  recommended.  And,  in  a  more  general  area  of  concern,  the  SPI 
effort  took  place  in  the  context  of  other,  broader  corporate  change;  the  PCM  did  not  address 
how  to  integrate  the  change  effort  it  addressed  with  other  technology  or  organizational 
changes.  Some  general  guidance  in  this  area  might  be  helpful. 

As  we  look  back,  we  are  once  again  convinced  that  this  type  of  collaboration  is  invaluable  on 
a  number  of  levels.  During  the  collaboration,  Xerox,  specifically  PSG  West,  learned  as  an 
organization.  This  learning  is  observed  and  documented  by  the  SEI,  which  in  turn  shares  this 
learning  to  the  software  engineering  community.  Other  organizations  then  build  on  this 
learning,  and  many  people  and  groups  benefit.  However  difficult  our  work  was  at  times,  we 
were  always  encouraged  by  keeping  this  larger  goal  in  mind.  We  hope  the  reader  has  found 
our  story  at  least  as  enlightening  to  read  as  we  found  it  to  write. 
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